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The themes of the i964 'Inyitational Conference on Testinf . Problems^ 
ifofcused upon varioui ^p^cts of testing within society /and upon 
technical advances in measurement A report in Session ^ on the effect 

testing pn the Chinese empire proylded an interesting contrast wifli 
a' paper on the effect of testing on the individual in modern America. 
Session HI was devote^ jto the prpblems of testing across cultures 
within a society and was explored from the yiewp^nt of the educator, 
the test-maker, the socidlqgist, /and the psychologist. In contrast* to 
thaie rather broad aspects of measurement, Sesilon 11 featured a dis- 
cussion by three leading psychometrifians of some new mathematical 
approaches to the analysis of test data. All three sessions resulted in 
a program that was both lively and absorbing and altogether appro-^ 
priate for the twenty-fifth anniversary of this annual event 

The success of tKis program was due largely to the fmaginative 
planning of the chairman, Professor Chester Harris, to whom we are 
all most gratefuL We owe bur thanks also to Professor Omaf Khayyam 
Moore for his luncheon address on 'Technology and Behavior," and 
to the other distinguished speakers whose, papers we ^ire pleased to 
publish in these Procegrf/Vig^. ' . , 

. ' r ^ Hemy Chaunvey 

PRESIDENT 
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^ / y ' it was m£'pleasujr|f to btf Invited to pian the<wenty-flfth JTivU^tional 
\ / '\ ConfcrenQe on Testing P^blernSiSponsor^^^ Educational Testing 
Scryye. iTo help^e determine what might^e appropriate papers and - 
• topics for discussidh^t tlii^onferen"C< 1 turned to a number of per- 
' ^ sons throughout ^Untry for adyicci responses were prompt 

, .vand^tremeiy helpful; in ftilt' f ifed little fflore^ta do than to select 
md then order into a reasorf^fr^equence a limited number topics 
„^ . ' . from jh^^wealth pf ideas. / received, (The sugge^fions for tony that 
we were not abffi tc^crWd into a^6J^e-Jiay\m 

and passefl on to future chairmet]^) Jhiis, t^ character, of tHis par- 
. " * ticular prAftm in 1964 was sha[5m largcly^by a numbeft)f other per- 
* " ^ sonSj to wnom. I am indebted, ' , ' , ^ 

/My second task; was to securp the. speakers. Again, hiy cprjespond- 
Ints fed hfen extrcmdy helpful in V^entifying for \me exactly the 
right person for . each ot a numfecr of iopics. I then wenl.to work to 
' secure acceptances 'frpm^tiese *'righ{'' persons/ Thei4^articipatiori 
gave the conference itj^ength; to them 1 am deeply fndS'ted, 

My third task wag^tq pfc^ide the wo tnCrning sessions and at the 
' afternoon .^sess|on.iThe interested audience Ksponding\^ interesting 
an4 provocative ppiakers made these duties especially e^oyable'. Thus, 
final expression of Indebtedness, m to the 'many whs came to herf, 
^ Th# firsts morning session presented two quite ^ifferent R^^^^ 
* dealing with^the effects f)f testing.. The second morningiession wa^^e- 
voted^o three somewhat techniq^r papers that seemed to me to repre- 
sent ^^eumportant^ and fairly recent^ Fines of development that un- 
doubtedly will influence our analyses of test data. Dr, Henry Chauncey 
presided at flie luncHebn mdeting and introducod Dr, O, K. Moore, 

- \ ' , J . , . page V 



v^lu) spoke on ;*Tichnoi9gy aiid Behavior;* the afternoon se^ion 
i mcluded four papers pealing with various issues "irT ihf prdblefti of ; 

"^^ce the record of the twenty^fth Invitat^nal Cotif erence oirTest- 
/mj'^Prpblemi is printed here for all to read.Mt is unnecessary tp'say . 
more. men speak for themselves; I hope they wiU apeak to you. ; 
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Philip H. DuBois ^ ^ . ^ 

Wash'mgton\jnivmity ■ 

Our negatwe enthusiasm for the present government in Peking should 
not lead us to a lack of appreciation for great Chinese achievements 
of the pastj' They have b^en many. ^ ^ , 
^ It is often said that the Chinese 'invented gunpowder and, quite 
humanely, used it to frighten, rather, than to kill, their enemies. 
^ Certainly' they solve J the V problem of diverse languages with the 
remafkable invantion of a common writfen language — a code by 
whic^ peoples who could not commimicate with one an&ther orally 
wer^able to communicaEe:freety .by means ©f writing. This invention 
was so suqfessful that the Chinese came to regaEd themselves as a 
single people. 

They invented paper, which the West did not know how' to make 
until some bhinese papermat£rs were caplujed by Arabs at Samar- 
kand in 751 A.D. They invented printing. They devejoped the arts. 
But, more importantly for pur purposes this Wrning, they indented 
the psychological test, applying it to government, the very framework 
of their society, in'such a manner that the test-makers, in effect; deter- 
mined over many centuries much of the Jormat of Chinese society. 

The prolonged and intensive Chinese experience with testing seems 
to have been completely ignored by contemporary psychometriclans. 
In none of the writings on gsychometrics with which I am familiar is 
thete any mention of some 3,000 years of examinations in the Chinese 
empire. This is rather surprising because' in clvk service procedures, 
ii is easy to trac^ the continuity of Eastern-arid Western^ methods. Con- 
tinuity between Western educational and psychological examining 
mpthods on the one hand and Chinese civil service testing on the 
other is 'more difficult to demonstrate, but some influence is probable. 
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Even if Western psychometrics hacl been complefely independent' 
of Chinese testing, the Eastern experience would^ have been of great 
interest to us. It affords the one historical example of a society in_,- 
which^xamining methods introduced to attain ccrfhin restricted objec- 
tives actually began to determine many characteristics, of the society 
itself; . ' ' ^ _ 

^ It should be noted that through the, liges the Chinese empire, unlike 
j^West» did hot. have a numerous hereditary aristocracy to consti- 
tute its'goveFning class. The chief wayio a political career was through / 
pa^ingia series of examina|ions in which coinpetition waivcry ^vei^. 
Moreover, China lacked another invention of the West: the univer- 
sity, ^e learned Chinese was'ope whahad been successful iu passing 
^competitive, cxamlrialions, arid whose success brought chaxiges in hij^ 
attire and in^Kis title as well as public recagnit1c)|i of his abilities, and' 
employment in government service. \ ' 

For long periods of time the systcm^vorked very well indeed. Only 
occasionally were cxaftiinations suspurided, one notable period being"^ 
the time in which the Mongol emperors rulcd in Peking. (The accounts 
of Marco Polo, who spent a number of years in China during their 
rule, make m reference to the Chinese civil service examining 

procedures.) ' ^ 

/The Chin;ese scholar^cems to have been a reasonably successful 
public administrator. Public office was often distributed by lot among 
the mandarins \vho had passed three successive sets of examinations. 

Millions of men prepared for the tests, often far decades, and rela= 
tively few achieved final suceess. The ^selection ratio was so small 
that the tests themselves Wtnild not have had to be very valid in order 
to be useful That they Were useful is perhaps indicated by their long 
history arid by the fact that for many centuries, with relatively few 
interruptions, the government of the Chinese empire preserved inter= 
nal peace, provided security from many would-be invaders, and per- 
mitted a flowering of civilization that in many respects was far more 
advartSed than that prevailing contemporaneously in the West. 

The earliest development seems to have been a rudimentary form 
of proficiency testing; About the year 22()() B.C., the emperor of 
China is said to have examined his oihcials every third year. After 
three examinations, he either promoted or dismissed them from the 
service. There seems to be no record of the exact content nor of the 
methods of testing, but the precedent of periodic examinations, was to 
continue for many generations. 
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' ^ Philip H. DuBois • 

A thousand years later in 11 15 B,C., at the beginning of the Chan 
dynasty, formal examining proceaurcs of candldatess Jgf office were 
established. Here the record is clear. Job sample tests were used re- 
quiring proficiency i^ the five basic arts: music, arbhery, horseman- 
ship, writing and arithmetic. Of the five, at least two, writing and 
arithmetic, still havp validity for public office, Knowicdjc of a sixth 
art was also required — skill in the rites and ceremonies of public and 
social life. . 

It should be pointed out that this examining system, which was later^ 
to be centered u^n the Confuciari classics, was actually in existence ' 
long before the time of Confucius (55N478 B.CJ. 

While the procedures changed from time to time, and the sources to 
which I have had access are somewhat coatradictory, a few dates 
seem to be clear. In 4 65 B,C,, by which time Confucian ethics had 
become current, moral standards were introduced in the selection of 
competitors. District magistrates were required to send to the capital 
candidates who had acquired a reputcubn for filial piety and integrity. 
Those whose moral character had been sufficiently attested were then 
examined with respect to their intellectual qualifications. At this time, 
tfie test included not only measures of the six arts, but also familiarity 
with the geography of the empire, civil law, military matters, agri^ 
culture, and the administration of revenue. 

After 622 A.D,, open competitive examinations took place at more 
or less regular intervals. By 1370 A.D. three levels of examinations 
were well established. The candidate who passed the examination in 
his district became eligible to take a test at the provincial capital, and 
those successfurat the provincial eapital were eligible for final exami- 
nations in Peking. For.about 500 years the system was stable and a 
description by William A. P. Martin in 1H70 is pertinent: 

... The candidates for office, — ^ those who are acknowledged 
as such, in consequence of sustaining the initial trial, — are divided 
into the three grades of siu-tsai, chu-jin, and tsin-shi, ~ "Budding 
Geniuses,'' "Promoted Scholars," and those who are '^Rcady for 
Office/' The trials for the first are held in the chief city of each 
district . . . They are conducted by a chancellnr, w^ose jurisdiction 
extends over an entire province, containing, it may be, sixty or 
seventy such districts, each of which he is required to visit once a 
year, and each of which is provided with a resident sub-chancellor, 
whose duty it is to examine the scholars in the intervaK and to have 
them in readiness on the chancellor's arrival 
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Abourtwo thousand competitors enter the lists, ranging in age 
from the precocious youth just entering his teens'up to the vener- 
^ able grandsire of seventy winters. Shut up for a night and a day, 
each in his narrow cell, they produce each a poem and one or two 
essays on themes assigned by the chancellor, and then return to 
their homes to await the bulletin announcing their place in the 
scale of merit. The chancellor, assisted by his clerks, bccupies^ sev- 
eral days in sifting the heap of manuscripts, from which he picks 
out some twenty or more that are distinguished by beauty of pen- 
manship and grace of diction. The authors of these arc honored' 
with the degree of ''Budding Genius," and arc entitled to wear the^ 
decorations of the lowest grade' in the corfwratlon of mandarins. 
The successful student wins nO purse of gold and obtains no ollice, 
but he has gained a prize, which he deems a sufficient compensation 
^or year? of patient toil. He is the best of a hundred scholars, ex- 
empted from. liability to corporal punishment^ and raised above 
the vulgar herd , , . 

Once in three years these "^Budding Geniuses," these picked men 
of the districts, repair to the provincial capital to ejigage in com- 
petition for the second degree, — that of chu-jhu or 'Tromoted 
Scholar/' The number of competitors amounts to ten thousand, 
more or less, and of these only one in every hundred can be ad-, 
mitted to the cuveted degree. The trial is conducted by special, 
jcxaminers sent down from Peking and this examination takes a 
wider range than the preceding. No fewer than three sessions of 
nearly three days each arc occupied instead of the single day for 
the first degree. Compositions in prose and verse are required, and 
themes are assigned with U special view to tasting the extent of 
reading and-depth of scholarship of.thc candid^ites. Penmanship is 
left out of the account, — each production, marked with a cipher, 
being copied .by an ofVicial scribe, that the examiners may have noJi 
clew to its author and no temptation to rendfcr a biased judgment,/ 

The victor still receives neither bHice rf^. emolument; but the 
honor he aghievcs is scarcely less than thatkWhich was won by the 
victors in the Olympic games. Again, he is one of a hundred, each 
of Ji^qp was a picked man; and as a result of this second victory 
he goes forth an acknowledged superior among ten thousand con- 
tending schojars. He adorns his cap with the gilded button of a 
higher grade, erects a pair of lofty llag-stafTs before the gate of his 
family residence, and places a tablet over his door to inform those 
who pass by that this is the abode of a literary prize-man, But our 
'Tromoted Scholar'' is not yet a mandarin, in the proper sense o' 
the term. The distinction already attained only stimulates his desir: 
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for higher honors, — honors which bring at la^t the solid recom- 
pense of an income. * 

In the spring of the following year he j^roceeds to Peking to seek 
the next higher degree, the attainment of which will prove a pass- 
port to offlcemiis contest is still with his peers, that is, with other 
''Promoted Scholars,'' who like himself have come up from all the 
provinces of the empire. But the chances are this time more in his 
favor, as the number of prizes is now tripled, and if the gods are 
propitious his fortune is made , , , If his name appears among the 
favored few, he not only wins himself a place in the front ranks of 
the lettered^ but he plants his foot securely on the rounds of the 
official ladder by which, without the pregtige of birth or the sup- 
port of friends, it is possible to rise to a seat in the grand council 
of state or a place in the ImperiaP Cabinet. All this advancement 
presents itself in the distant prospect, w^ile the office up^n which 
he immediately enters is one of respectability, and it may be of 
profit.lit is generally that of mayor or sub-mayor of asdistrict city, 
or sub-chancellor in the district examinations, — the vacant posts 
being distributed by lot, and therefore impartially, among those . 
who have proved themselves to be "ready for office/' I 

Before the drawing of lots, however, for the post of a magistrate 
among the people, our ambitious sludent has a chance of winning 
the more distinguished honor of a place in the Imperial Academy. 
With this view, the two or three hundred survivors of so many con- 
tests appear in the palace, where themes are assigned them by the 
Emperor. himself, and the highest honor is paid to the pursuit of 

* letters by the exercises being presided over by his Majesty in 
person. Penmanship reappears as an element in determining the 
result, and a score or more of those whose style is the most finished,, 
whose scholarship the ripest, and whose handwriting the most ele- 
gant, are drafted into the college of Hanlin, the "forest^of pencils," 
a kind of Imperial Institute, the members of which are recognized 
as standing af the head of the literary profession. These are con- 
stituted poets and historians to the Celestial Court, or deputed to act 
as chancellors and examiners in the several provinces. 

But the diminishing series in this ascending scale has not yet 
reached its final term. The long succession of contests culminates 
in the designation by the' Emperor of some individual whom he 
regards" as the Chuang-Yucn or madel scholar of the empire . . . 
Provinces contend for the shining prize, und the town that gives the 
victor birth "becomes noted forever, SwifQeralds bear the tiding^of 
his triumph, .and the hearts of the people leap at their approach. 

. We have seen them enter a humble cottage, and amid the flaunting 
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' of banners and the blare of trumpets announce to its startled Inrfiates 
^ that one of their.relations' had been crowned by the Emp^rqj^as the 
' toireate of the year. And so high wa^ the estimation inpl^ich the 
^ people held the success of their fellow-townsman, that m wife ^as 
Requested to visit the six gates of the city, arid to scatfec before 
eabh a handful of rice, that the whole; population migjit share m 
the good fortune of her household - .\ ^ 

It IS ol^ious that which otcltes so profoundly the interest ot a 
whole nation iTiust.be productive of very decided results. That it 
.leads to the selection of the best talents for the service of the public 
>ve have already seen; but ^beyond this — its primary object— U 
exercises.a profoiyid influence u^n the education of the pcpple and 
the stability of the governinent/U Is all, in fact, that China has to 
show In the way of an educational system. She has no colleges or 
universitics/^if we except one tlpt is yet In embryo,^ and no 
national ^mm of common schools; yet It may be confidently as- 
serted that China gives W learning a more efTectivc patronage than 
she could have done if each of her emperors were an Augustus and 
every premier a Maecenas. She says to all her. sons, "Prosecute ' 
your studies by such means as you may be able to command, 
whether in pubUc or in private, and when you are prepared, present 
yourselves m the examination halL The government will judge of. 
your prdficiency and reward your attainments." 

Nothing pan exceed the ardor which this^i^tanding offer infuses 
into the minds of all who have the remotest prospect of ^har;ing in 
s the prizes. They study not merely while they have teachers to mcite 
them to diligence, but continueThcir studies with unabated zeal long 
after they have left the schools; they study in solitude and poverty; 
they study amidst the cares of a family and the turmoil of business; 
and the shining goal is kept steadily in view until the eye gmws dim. 
Some of the aspirants impose on themselves the task of writmg a^ 
fresh essay every day; and they do not hesitate to enter the lists as 
often as the public examinations recur, resolved, if they fail, to con= 
tinue ^trying, believing that perseverance has power to command 
success and'cncouraged by the legend of the man who, needing a 
sewing^needle, made one by grinding a crowbar a piece. of 
* granite. 

This x]uotation from Martin, describing and praising the Chinese 
testing system, is by no means unique. The use of competitive examina- 
tions forjhe selection of state ollicials was praised by many Western 
observers and writers, including Voltaire. In fact, it is cleai^ that 
initially all civil sefvice examinirig in Europe und in the United States' 
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used the Chinese syst4m/^directry or indirectly, as a modcL Civil. 
m service testirig was introduced in France as a 1791 .retom, only to 

be abolished by Napoleon. In England the first competitive examina= ^ 
. tions in connection with public office were anstitutpd for the selection 
of trainees for the civil service in India By men familiar with the Chi^, 
nese system! Later, when the. question of civil service eHarfiinations 
/ for Great Britain, as a whole was debaterfTin Parliament, th^ Chinese 
model was discussed with, both favorable and unfavorable- comments. 
As^ part of an extensive, study, Congresynan Thomas A. Jenckes, , 
A " one of the fathers of the United States Civil Service, wrote 12 pages 
on thefcivil tervice of China. 

Westerners seem to have been particularly impressed with the fact 
j ^ that cdmpetition was open, tbat distinction cam* from iperit; and that 
^ a highty literate. and urbane group of public offici^als resulted from the 

examination syste^i, - ' \ ■ 

The'great^ikis inCh nese affairs came, of course^ when the Chinese 
ey were igj^tarily inferior to the West. They quickly 
equality in militafy power could not be achieved with^ 
out i^^ri siiendB and/technology. Accordingly, technological schools 
and utUversit^s were afet up, but as long as tht civil service cxamina-- , 
tions, which were largely literary in chEjracter, continued to be^the 
-way for ah ambitious man/to have a carcbr, moflern education. was 
not suffictently attractive. Consequently, in 1905, the Chinese cxamir= 
nation system was abqlished^as a reform measure. 

So much for a description and a bit of tl!e history of an ancient ■ 
Chinese venture in psychological examining as a tool of government-,^ 
What can be said about their testing techniques from the point of 
view of the modern psychometrician? In the hrst place, I find no evU . 
dence to indiffite that they invented either the multiple-choice format, ^ 
the test-scoring machine, or item analysis. They did, however, recog=^ * 
nize that a relatively' short performance under carefully Controlled 
conditions could yield an estimate of the ability to perform under less 
rigorously controlled conditions and for a longer period of time, 
I think there is no doubt that the procedure selected capable public 
servants. 

They recognized the problem of objectivity, concealing candidates' 
names and sometiipes using a bureau of copyists to copy examination 
material before it was gradcfl. in some crises, tests were read by two 
independent examiners who handed their scaled evaluations to a third 
examiner who reconciled any diflercnces. Scores seem to have been . 
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In terms c^^rank order, ^ * * , 

The need*for uniformity in testing conditipns was well recogpized. 
Considerable attention v^ss given to procuring the examinatioji halls, 
vihich werr large and permanent Installations consisting W hundreds 
of small Cells, Sombtmies candidates died during the rigors of the 
examinations, which went on day , and night. 

In this yea^l964,twhen psychological tests are being dscd more and , 
more extensively at critical points in the careers of all our citizens* 
we will do well to consider their eff&ts on individuals, on specific insti- 
tutions, and on 'society. The' long Chinese experien^ Is a pertinent 
c^e history /It is a plausjblehypofliesis that much ofotl^ grealltrength 
of the Chinese Empire came frpm the intellectuaT vigor of men who 
were- bright enough to compete in cMminaiipns requiring the writing 
of poem§ and "jight-legged cssays^y ^ 

Certainly the opportunities that wci^ qpcnetinip by success^^ ,the 
examinations stimuiated millions of individuals te' long years of schoU 
an^hip. Perhaps the ^greatest drawback was that the scholarship was 
not always pertinent. In the nineteenth cen^tury, China suddenly found 
herself surpassed in technology by the West. While Chinese civiliza= 
tioh had' been ^relatively static. Westerners invented the steam engine, 
the power loom, and the ironclad, was tfien that the Chinese, in 
order to preserve their country and their institutidns, began to desire 
progress according to the Western modeh At that time the age-old 
examining system was discovered to be a hindrance. 

So far, with 60 years of experieace. sve Westerners have not found 
our psychological exaiTiining a hindrance. But it is becoRflng increas- 
ingly apparent that otir test-makers, like those of^cient China, 
establish goals for. individuals and inlluenee the shape of sociabinsti- 
tiitions. Item writers as well as song writers *mald the patti:ms of a. 
culture; . ^ 
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This stjudjrorigfMted in a raertiodological quc|ton. pperatiqpis f^r j 
measuring pefsonaJity^ are less satisfactory thanthose for assessing 
abilities and aptitudes. Does this re^^itive madequac^stem fTOn the- 
heterogeneity of^iubjects' reiiotions to tests? More^pecificully, are 
reactions .to personality tests ffiore diverse ^an those eognitive ^ 
tests? If so, doel such greater df\ttsit)t^ help to accoiiat for this 

inadequafcy? * ^ . ^ ^ 4 ^ ^ u 

The initial purpose was to explore reactions to tests by medns of the 
simulated subject technique which Orne (1962) developed to evalu- 
ate the adequacy of the experimental conditions. used in some areas ^ 
of psychological resqarch. Jo permit greater generalization of the find- 
ings/.the surve/ inethod was selected. During the planning stage, it^ 
became obvious that other significkt information could also be ob- 
tained at little additional cost. The actual study sought answers p hve 
basic questions _ . 

' I. What experiences hav^ Pf^^^ ^^^^ c^ntry had with psycho- 
logical tests? * • 

• 2. What view^^ about psychological tests have they encountered? 

3, How do laymen evaluate psychological tests of^ various kmds? 

4. How do people feel about taking tests? (How do" they feel aboUt 
being given a test? Haw do they feel Uhile taking it?) 

* This^study was supported by Grant mH-06582 from the National InMitute 
of Mental Health, United Stales Public Health Service, 

I am indebted to Naomi Bernev Jere Brophy, Castenano ^^urner, and 
Thomas Tyler for their assistance in the ^atistical analyses, I am also in^ 
debtedHo the staff of the National Opinon Research Center, especially Carol 
Bowmiti, Gilten Gockel, .Harold Levy.lind Patrick Page. 
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5, What are* the methodological implications of the answers to 
these fo^r questions, especially the latter two? 



Method 



1%e "study qf these ^questions was conducted with the assistance of 
'the National Opinion Research Center whose excellent organizatfon 
condupted'the inte^vjews and did the coding and tabulating. A 



^ tion^ sample of 589 respondems was obtained, It^was limited to 
^/ persons 21 through -64 years oF^q who had completed .at least, the 

, sixth grade.^*Tfie sampling was intentionally biased to obtai^i about ^ 
one-third- more respojijdents who'had boun to college' than would be' 
found m the [national populations there were 27 per cent|in^thal 
catego^ rather than'20 per cent, w . . i 

Hie sample Vas quite representative of the country at large with 
respect to sex, race, and' age. The ' correspondence on place of resi- 
; dence and on'o&cupalion&hgroup was cte^ 

approximate ^neralization* t6 the national popuhition, 

( The interview^generally took about an hour. The questions asked will 
f. be, brought out in;the preseniatibn of ^rt^Jridings, However, the simu-^ , 
laled subject procedure requires detailed deshription. This part toot > 
' f. place during the middle portion of the inter^w. ^ ^ ^ - T ' 
' Each respdhdent was- assigned in Midvanqe to one context and to * 
two tests. The* Job context had instructions asking the subject ^to sup- 
's* poA that he whs applying for a job^and that he^was given a test. This 
^ , supposed test started with the instructions and the several items that 

were actualiy presented to him; he was ?isked to respond to these i 
" litems. In the Research context, the instructions asked the subject to 
suppose that a college professor had^ asked him to take the test for 
^ . vibme research he was doing, and to answer anonymoudy. The con- 
text was referred to again when the second test was givert; it was also 
-^^ - mentioned twice during the exploration of his reactions to each test. 
The six incjomplete, simulated jests had titles and instructions 
closely resembling those from which they sverc borrowed. The six 
included the foHowing: an intelligence test based on the ACE^-an 
interest test derivcd'from the Strong; an innocdous personality ques= 

' 22 ' 
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# ' ^ * ' I* ^ . 

.iionnaire adapt^ from the Thurstone Temperament Schedule;' a less,^ 
* ipnocuous- Multiphase Inventory based on the mmpi; an Incomplete : 
%ent2ijcei ;^t; and an instrument with two ink-bloft. The first Ave 
% ' had sWin'jflileven items eac^^ „ 

j^^^dard mi toj thesignificance-of differences m proportions 
^l^l^l* tlf .^mparison was between two parts which together 
^l^^^^^'-totk /sample. For comparisons of pairs within a set of 
he%ll^o^fti^^the sludentized range test provided th& criterion 
for each comparison, with Fecognition of th^ relative severrty of this 

standard.,)*' . _ 

Thefliethod appears to have worked fairly welU Since it seemed to 
m^e little diflfcrence whether a test cajne ft^t or second, all respond- 
. ents given each test within the sanie contex^ were^, grpuped together. 
^ Thus we had about one hundred\subjects p^^test per cantext. NoU 
withstanding the' arbitrary assignment to \hese ^perimeni^l condi^ 
tions, the six groups within each context were rea&nably comparable'' 
on demographic variables, as were ^ the two main context groups 
themselves. 

The experimental technique appears to have been fairly effective. 
This impression is supported by internal evidence, such as tine direc^ 
tion of group d^erences: Also, the subjects reported that they main- 
tained the instructional set for the specific context fairly wcli. As 
might be expected, however, a handful of subjects did objec^to their 
being asked to take the tests. The level of affect aroysed in the sub- 
jects seems'to havabeen below that which the subjects recalled having 
experienced during their last/ actual test. 



Findings 

Expsrlaness with Tests and 6vatuatlans of Tests 

Almost two-thirds of this sample of adults reported having taken at 
least one test, other than tests taken while in some schooL Half of the 
total group had, had a test when applying for a job. Onl>^ 15 per cent 
had taken tests for admission to college, these comprising a little over 
"^'half of those who went to college. Of those going to other schools, 
smaller proportions tbok tests for admission* 

Exposure to views about tests was reported by 62 per cent. Many 
remembered having read something, some had heard discussions of 
tests, an*d a smaller number had taken courses in which tests were 
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presumaSly mentioned, The views recalled were*fnore favoFabla than 
unfavorable. i 

When a|ked to select the best way to find out whether a person is 
qualified for something (^cfi as a job or admission to a sQhQpJ), the . 
interview method received most first choices, especially among thdse 
with the most college eduction! Tests came second, receiving one-, 
third of the first choices and one-quarff r of the second choices. AppH- ^ 
^ cation blanks and references were preferred by much smaller propor- 
tions of the sample. : 

Respondents were given fou^choices, from very good to very poor, 
to intiicate their Valuation of tests for finding out about a' person's 
abilities and skills^jp likes and dislikes, how he gets along with other 
gpople, and what his. pergonal probkms are. Jjte results are given in 
Table 1, More than 75 per ocnt of the sample evaluated tests as good 

f ^ ' : , ; ■ / . ; 

table 1 ^ , ^ 

Ev^luatlsns sf Tmmtm for Different Charaof#rlstlcs 





(In Per.depta) 








How good sr pQor ds ysu think 
tests WQuid be for finding Qut^ 


V^y 

Good 


Fairly 

Good ' 


Fairly^ 
Poor 


Very 
Poor 


, , f , whst o person's aptitudeif 
skilhf and ebUitisi sre? 




51 


10 


5 


. , , , wh^t a persQn Ukpi on^ 
dislikmif 


35 


44 


10 


) ] 


. . , . how Q person gsh along 
With other peopfef 


23 


30 


23 


24 


, , , , what someone's psrsanal 
probhmi ore? 


20 


28-^ 


22 


30 



for assessing abilities and for muu&Uring litcs and dislikes. In contrast, 
only about half rated tests as good for finding out abouf a person's 
social relations or personal problcitis. I suppose personality testers 
can be pleased that the proportions, are even that large. 

Interpretations of Tests 

After these general opinions had been explordd, each subject was 
given two simulated tests. The final question asked about each was: 
"What do you think that^est is supposed to find out?'' The replies are 
cjassified in Table 2, 
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^^©p!l©s to the Qyestlani 
^Vi/hat d© you th^nk that test Is aupposed to find out?" 
' \ Cin Per^Cents) * 



J- " ^ . 

tntelllgefi€6 
kn&wtedie 

Verbdi ikillf 
Peripnallfy 
Sfsbillty 
getting Qlong 
with others 
Aspeciflc trait 
fntereifi 
. IrnqginQticn 
Multiple answer 
trrelevsnt and 

miicettqnedyi 
Don'f knew 



"Fiyehologkal 
Ixqmlnotldn'' 
(intelligence) 

\ 

\7 ■ 
W 

R 

A 

Oi 

i. 

; q\ 

21 ' 

7* \ 
17 



Simulated Test 



"Preferenge 
Inyentery"' 
(jntereits) 

2 

1 

2 
0 
6 
1 

1 

51 
19 

6 
10 



"Uk^ and 
Dlslikei" 



"Multiphgsie 
PeriOnsllty 



{Temperament) Inventsry" 



0 
2 

0 
25 
4 

4 

3 
8 
0 
27 



15 



0 

"o 

31 
14 

0 
0 
5 
0 
29 



10 



Incomplete 
Sentences" 

12 

2 
4 
4 
24 

1 . 

1 

1 

2 
0 
24 

10 



"Ink* 



Il0t4** 

12 

3 
3 
0 
15 
5 

0 ^ 

0 

2 

17 

14 

)" 
6 

23 " 



The majorityj hazarded speciiic guesses and thcne are the ones with 



he concerned. The intelligence test was interpreted as 
jritv of thostf giving specific replies. However, a few 



which we shall 

slich by a majo|rity of thos# giving specific rcpl 
thought that it jneasured personality! -Tlie interests test received an 
even clearer concensus of correct judgments. The so^nUled "Inventory 
Of Likes and' Di^likes*\ (borrowed from the Thurstone Temperament 
Schedule) was given iporc varied interpretations, with personality 
being the modal! specific response, A similar pattern of perceptions 
was obtained fo^ the Multiphasic Inventory, but stability was more 
frequently mentioned for this test. The same general pattern recurred 
for the Incomplete S^nces, although intelligence and verbal skills 
received more mentions and stability received almost none. While 
the modal respo^ttbr the Ink-Blots was imagination, almost as 
many saw it as measuring personality and, like, the other projective 
test, a numbet thought it measured intelligence; 
' It is obvious that the ^mnjority of those giving specific perceptions 
of these tests made interpretations which" those who developed the 
tests would* consider highly appropriate and accurate. Not even the 
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|eneral_ purpose of prdjecUve' tests cc^d^ considered as disguis£d.% 
Examination , of the s^ulate^^ tests indicates that the respondents 
ba§ed^their interpretations primaffly on the content of the items, 
rather than on the names of the Jests or the instructions. \, 

ResGtioits to Simulated Tests 

After each subject completed the brief simulated test, he v/al asked 
* how he would feel about takingMt the Job or Research' context 
which had been portrayed to him before jthe testing. This op^n^ende'd 
Question elicited a diversity ol reactions. A' qmirtcr pf th|^s|"nlple 
. expreped indifference. In. additidii, Tilmost half the Job group aijd 
more than % quarter of the^ Research group cxprej^scd some kind of 
Qcgative reaction^ with criticisms of fhc test predominating. Veryjpw' 
expressed uneasiness.pr discomfort; very few said tests are prying or 
can be^Jaked. (For each context group, less than 5 per cent of the 
replies fell in each of these latter four categories,) , 

In the attempt to uncover hicfden negative reactions,; respondents 
were also asked whether other people would feel differently about 
taking the test, and if so, how they would feel Whilgjlclpse to 10 per 
cent projected some kind of negative reaction, the n^ajority of those 
sayfng that others^ might feel differently had the good ps^^chological 
sense to observe that it would depend upon llic person,. Although the* 
layman's psychological understanding is naive and untutored, wc once 
again find that it is rather sound. 

Nomothetic v^ata on reactions werd also, obtained by asking the*' 
subjects to ii^cate which of 13, phrases described and which did not 
describe how they felt whilp taking the test. (See the first two columns 
of Table 3.) ''DiHn't mind,'' ^^nterested," and '^Curious" were se- 
lected by majorities, In'addiiion, denials of most of the phrases iiidi= 
calihg negative feelings were made by majorities. 

Of the many comparisons beuvet^n the Job and the Research con- 
text groups, only a few were significant. These are judged to he real 
differenees in most instances since thoy tend to be consislcnt and in 
the expected direction/ For example, the Research group expressed 
mere willingness to cooperate. More of the Job group comsjented that 
the test had a poor rehiiion to job performance and gave negative 
comments of other kinds. But alsoi mo^ of the Job group said tests 
were good, (Once again, the reUnivc frequencies arc snftall and these 
findings show diversity wifhin the group rather than inconsistency 
within the individuaL) ^ . ■ ^ 



Sy&J«st8^ Fsetlnga while TaNrng Ttists 
(In Per Cents) 

, ' Nellngi While Taking Reesllertlsn of Fasllngs While 
ilf^uleted Test Tskfrig Lsif Agtusl Teit 

' . ApplUoHen for . 







Job • 


Research 


Appllsotlon 


^ Admission t 






Context 


Context 


for Job 


School or Coll) 




^Deiaibfj 










\ WQi /nfertiM 


Dpei 


59 


64 


69 


74 




D^s not 


14 


10 


7. 


4 




- Po«^ 




61 


35 


52 




Does not 




10 


17 


13 


1 felt nn^&mfoFfMs 


Doei 


12 


10 , 


2S 


24 




D^^t not . 


53 


60 


34 


37 


1 didn'f mind 


Poet 


69 


75 


42 


44 




Does not 


12 


6 


19 


19 


1 Wdi oniiojfed 


pqes 


6 


3 


11 


^ 9 




Does not 


61 


67 


53 , 


56 


1 felt Hka tavghiftg 


Does 


17 


16 


2 ^■ 


4 




Ddes not 


49 


54 


65 


61 


1 wdt dfijcfowf ^ 


Does 


12 


10 


55 


54 




Does not 




46 


16 ^ 


19 


J wds irrif of ^ 


Does 


4 


2 


4 


4 




Doei not 


6S 


70 


51 


54 




PpBi 


5 


2 


2 


4 ' 




Doei not 


63 


70 


63 


63 


1 felt wf t iiity 


Does 


25 


14 


6 


11 




Doei not 


44 


56 


63 


' 54 


1 WQs^ffyttfs^ 


Does 


5 


4 


S 


9 


Doei^not 


55 


62 


46 


44 


1 was fenit 


Does 


8 


10 


56 


4S 




Does not 


, 57 


60 


19 


23 


1 felt p/eeied 


Does ^ 


23 


27 


27 


24 




Does not 


28 


24 


25 


20 


N 




.(297) 


(292) 


(259) 


(54) 



Reactions to the several pairs of tests were examined separately for 
-,€aQh eOTtext, Few significant differences were found and in no case 
was the same pairing significanf in botli contexts,. Since the statistical 
criterion may have been too stringent, an j since there may have been 
mteraction between test and context, some of the significant differ- 
ences will be mentioned. Most involved the Ink-lMots Test. More 
__Qftig thgp Jp^ gt^^ ^sts, it was seen as ridiculous and it made sub- 
jects feel embarrassed. The intelligence test was more likely to make 
subjects feel uneasy and the Multiph^ic Inventory elicited more com- 
ments that tests can be faked/ 
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Aftir toe simulatid tests, ttie subjects were asked about their expe- 
nm^ m^mtu^mt^ most recent actual test was identi- 

ifdrpbr mai^, this was rtest taken in school many years ago. Even 
for ftose taking tests when ap^ylng for a job, the median time interval 
was six yeMS. Therefore, the reported feelings about these last actual 
tests must be seen as filtered through years of memory. Reports Jor 
tests taken when applying for a job and for admission to a school or 
college ite. ^yafl liylje last two columns of Table 3= 

G^^anted this limitation stemming from remoteness in .time, the de- 
scripliions of feelings on the Airteen items show a number of striking 
d^erences from the feelings about the simulated tests. With one minor 
eSepUon, for each of the several real contexts, a majority of the 

^subjerts'Te^rted feelinf- anxious and ^ tense; these proportions are 
several times those for the simulated tests. Reactions to the actual 
tests more frequent!)^ included "feeling uncomfortable and frustrated 
while tj^e feelings about the simulated test were more often "I didn't 
mind/' felt like laughing," and was silly.'' 

Thus, niuch more negative affect was reported, in long retrospec- 
tion, for actual tests. In comparison, while the free responses to the 

Ijueition about how they wouM feel about taking the simulated tests 
in the indicated context did produce tnore negative reactions than 
positive, these were ty|ically criticisms of tests rather than reports of 
ne|ative feelings. FinJny,'We have the reports of how the subjects 
felt while taking the simulated tests, reports which are more positive 
than negative. Thus, we find^i^easing amounts of negative reactions 
with increasing levels of potential consfquences, starting with how the 
subjects felt about taking^short, obviously inoompljete mock tests in 
the survey intervie^^ing on t^how they thought they would feel 
about taking a complete test in real life, and ending with how they 
recalled having actually felt while taking a test which often could 
affect theff lives. (Data fmm another group given a similar, but 
briefer, procedure are consistent with this trend: for the reports of 
these subjects on how they would feel about taking the simulated test 
in a" given context, the frequencies for "anxious" arid *'tense" are 
intermediate between those for the actual test and those for the simu- 
lated test in-the survey group.) 

On t^e one hand, it is clear that the simulation of context forked 

--to only a limited degree: tha^simulated te^ts were probably not suffi- 
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cient^ lialistie and the subjects were not given enough firip in the 
; role^kldni we triid^fo have them perform. On the other hand^sthe 

findings ^arly suggest that being tested, even in an artiflcial situa- 
'^^^m ^^ fE[^M'WP^^^^^^ the degree of impact in^ 

creases wift the potential consequences for the subje^^^ ^ 

MMhMloiogloal ImplloatlQns 

Our findings indicate ^at a person who is given a psychological tes^ 
willVBsually have a reasonably good idea about the general charac- 
teri^ic which is being tested/However, his feelings and reactions wHT 
be determined only in small part by the perceived nature of the testn 
rather, he wiU be reacting primarily to the fact of being tested. If ^ 
great deal depends upon his p^ormance, he is likely to feel anxious 
and-^ensa^ and quite unders^ndably so. But even if the test results _ 
will not affect him one way or the other, there is still a good chance ^ 
that he will feel uneasy, perhaps even resentfuL 

While such feelings do occur commonly, other feelings and per- 
ceptions^;nay also be present. The subject may have great faith in 
tests, oHie may see them as slHy and invalidphe may even feel that 
testing is wrong, tlien again, he may enjoy taking the test. The diver- 
sity of subjeets* "perceptions of tests and feelings about being. tested-^ 
must be considered in designthg the procedures for administering a 
test, and particularly in formulating the Iftstructions, ' 

Jn plnnning this research' it was anticipated that the diversity of 
Hl^ions and reactions would be least for the intelligence test, and 
would increase for the other tests, through the test of interests and the 
other questionnaircs to a maximum for the projective tests. These 
differences were not found. Hence the differences in psychometric 

^dequ^cy between intelligence tests and^ personality tests cannot be 
explained on this basis. It now seems that the critical difference is in 
the tasks involved. 

Tests o with a clearly defined task, one 

which he understands and which he is able and willing to perform, 
in addition, they present a set of stimuli, each of which is interpreted 

. by the several subjects in' the same .way, at least with respect to how 
their responses to it should be determined. By these means, tests of 

^ abillly nrtniifKizt t^^^^^^ intrusion of such irrelevant sources of 

variance as tfie diversity among subjects' evaluations of tests and the 
variltioir imong^their subjecti^ feelings while taking the test, 



page 20 



^9 / 



^ Donald ^ FIske 

the Other iide,*the task posed by the typical peribnality test is 
much less J^isely defined. The instructions, though intelligible, do 
not fumlih ttif subject with explicit criteria for selecting his response: 
^f^after mdctaf^W^ to review it, he could usually 

not be confldent that he had.given the response called for in the in- 
struatlons. In addition, if the test is Verbal, there is vagueness and ^ 
ambiguity in ft|e items themselves and in the permitted^response 
alternatives (e.g., s©S Benton, 1935-^ Eisenberg, 1941). 

This lack of itrueturlng and of consequent restraints in personality 
tests pemite the subjebt to perceive the task as a whole, and each 
item in pifctlculars in a way determined in part by his .reactions to the 
total testing situation. Hence tlje differential reactions found in this 
fiiearch can affect (he way in which a pifson responds to a test of 
this kind. We can expect that different subjects will approach the test 
diflfefenttyr If ^so,' the icorei from their responses wilhiot be 

.comparable because the experimental conditions have not produced 
similar constraints in the several subjects (cf. Fiske k Butler, 1963). 
*^ • ' , - 

Summary 

A majority of the adults in this country have taken a psychological 
test outside of their schoolwork, A majority have also been exposed to 
views about tests. Although tests Bre not of great concern to the lay- 
man, he makes highly appropriate interpretations of the purposes of 
tests, He typically evaluates tests favorably, especially for determining 
a person^s qualifications, for assessing abilities and skillSj and for 
determining likes and dislikes. 

This study found a considerable range of reactions tp the condition 
,pf being tested. The particular test is unimportant, in contrast to the 
effects associated with being tested. Negative reactions, and especially 
reports of negative affect, increase in frequency with the significance 
of the situation to the subject. . 

It is argued that tests with explicit tasks and definite criteria for the 
subject to use in selecting his responses can provide sufficient structure 
and constraints to minimize the unwanted variance which differenUal 
reactions would otherwise contribute to obtained test scores. 
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Recently I have been worlwg on two different kinds of problems 
which are not unrelated, One is substantive, dealing with certain data 
^intffiigenSlesS; t^^ The substaritive prob- 

lem has concerned me over, a number of years, A few months ago, it 
flnally seemed traetable, although incompieteiy solved. The data were 
.now orgamzed in.a certain fashion j and my plan was to present them 
in this manner/But'something happened after I arrived in Ann Arbor 
(to serve as Visiting Professor for this academic year). 

Just before leaving Jerusalem,. I had finally succeeded in working 
'out the basic equations for thd computing problem, providing a gen- 
eral nonmetrlc technique for defining Euclidfean spaces^ for data. This 
sounds very forbidding, but is actually very simple in its outcome. 
Small empirical examples were worked out on a desk calculator; these 
and the equations will appear shortly in Psychomeirika (6). Larger 
problems, like the substantive data of this paper, are more con- 
^veniently done on ah electronic computer. Dr, James Lingoes and I 
have been collaborating recently on developing an appropriate IBM 
7090 program. The week before this conference, the computer was' 
fed the data, using the new nonmetric program. The results were so 
^tartlings I tore up the intended speech and must present a differ- 
ent one! 

The general problem is that of the structure of interrelations among 
mental tests. It has, been tackled from several points of view in the 
past, one of which is the definitional aspect. For several years some 
^cblleagues and I have been working on a definition of the universe of 
^"ffieMtartbilitieir Recently we arriyed at a proposal, which is In press 
In Jerusalem (4). It reacte as follows: r 
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i9S4 Invitational Confartnee on Testing Froblami. . ' ^ 

• . , A Faceted Definition of Intelligence 

An ict of a subject Is 'm^eWg^m to the (extent) to which it is 
^.^ elasiifled by a (tester) as^ (demonstrating) a correqf^percepUon of 
,v ^ W. u a (relation) intended by the 

; tester; on the%sii of anrther (exhibited) logical (aspect) of that 
H ; relation that is corr^ily perceived by the subject. ^ 

' Israel is not as old as China; so it is not yet a "test-dominated 
society" so interestin^y d^cribed by Dr. Philip DuBois earlier this 
morhlng. But we are getting more ihvolved in testing. Two testing 
programs with which I have been associated have created^ for me the 
need to clarify a number of the issues and have led to this proposal. 
'A full explanation and exploration of the above definition could take 
several weeks or months. But we need grasp only a few aspects of it 

^ .for-our^purpQses-this morning. . . , 

, ^ This definition Js called ^'faceted-' because it is in terms of what ! 
call "facets/' namely, the parenthesized variables. In essence, it says 
that testing is a process of communication between the tester and the 
subject^ and that to characteriza an Intelligence test item we have to 
characterize both the stiniulus and th6 response. 

In working with pencil-and-paper tests; (as testers have for these 
many years) , if we ask how a tester can communicate with a sub- 
ject, the answer appears tof-^e that there are three basic kinds of 
"languages*' at his disposal, He can use words — the mother tongue 
of the subject and of the tester perhaps. He can use symbols of a 
more formalized language — such as arithmetic or algebra — where 
not just the symbols are Involved but the subject has been educated 
in the syntax for use of these symbols (e,g,, the subject knows what 
algebra is). Or he can use p/c/wres — whether objective or non- 

^ objective, These three languages can be used separately of in com- 
bination. It is hard to think of any other way of communicating by 
paper and pencil, 

These languages of qommunication constitute one importan.t facet 
of the means of ^'exhibiting logical aspects of relations," that is, means 

' of constructing items. Our point of view is that Jacets of this sort are 
guidelines for Item construciion. If you want to construct itemSj one 
thing to consider is: what language of communication should be used? 
Should It be verbal, numerical, or pictorial orSsome combination of 
these? 

Apart from the language of communication, one has! tq be"^:con- 
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eemid'^atout Qie substance being communicated. This again is an 
important facet of what is being exhibited, ^or of item construction. 
We have found it possible and u|eful to distinguish between ^'analytical 
abiUqr" itan^ and "acW^ itrais Just, by, making a crude 

dichotomy of the Kindt of thingi which can be exhibited about a rela- 
tion. There are two kinds of things: a relation has a rw/eand a relation 
has efcmcwij. Interpret the word **relation** here in the sei^e of the 
dieory of sets, of which we need no technical knowledge beyond 
recogniring the two aspects: a rule and the things which conform 
to the niter / ^ ' 

/ An malytical abijity item Is an\item in which the subject is asked 
to deduce the rule. You show him some^ of the elements which Qon\ 
form to the (unexhibited) rule, and he has to infer the rule froth these. 
For 'example, analogies are analytical ability items. You exhibit paired 
elements, ^uch as (cow, ?), The subject, in answering 

this properly by setting "calf in place of the question mark, shows 
that he has deduced the rule. Elements were exhibited here which 
^bey a^ruLe, but it was not said explicitly wkat the rule is; the tester 
Infers from- the response that the subject has a correct perception 
of the rule. 

In an mhievement item, the rule itself is exhibited and assumed to 
be understaGtd by the subject; the subject has to operate with the ele- 
ments properly according to that rule. If the subject is asked: "What 
IS the sum of 7 and ST' the assumption is he knows what addition is. 
The Intended problem is: can he work properly according to the rule 
of addition, that is, can he do what he knows he oughf to do? The 
question '"Who was the first president of the United States?" is an 
achievement item, assuming the subject knows what "president" 
means and what the ''United States" means. He knows what the rela- 
tion is — he knows the rule; the question is: can he come up with 
the proper element which obeys this rule? 

I have recently been exploring these ideas by looking over previ- 
ously published results of others on various batteries of testSs since we 
have not yet been able to design and administer large batteries in 
Israel. There are many rich treasures to be found in our litera^re of 
correlation matrices for large batteries of tests, but amonj the richest 
still are the excellent series of the "^hurstones, the very ones from 
which "primBry^ental abiliues" were first sought by simple-structure 
multiple-factor analysis (7,8). 1 had recently been looking at these 
large batteries again, and had thought to give you today some pic- 
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tur^ Of portions of their correlatlfan matrices, selecting put fl pr/ort 
the teste whieh clearly obey — or better, are defined by — the two 
faoeti^just discussed: the facet of lanpage oft communication (numeric 

li ning done witli 

the subitance being communicated (analysis or achievement). This 
gives a'^iix-fold classifica|ion of the tests, since each of the three ele- 
plants of the fim facet can go with each of the two elements of Uie ^ 
second facet. ' v 

The tests can va^ on other facets. For example, the subject can be 
asked ©ither io recall m to recognize an answer. An attempt was made 
to hold this particulai' facet constant by picking out only tests in which 
checklists were exhibited. A few exceptions were allowed when there 
were not enopgh tests to flll in an area. Tests calling only for memoriz- 
ing were cbmpletely omitted. Among verbal tests, the rule in some 
Hses ref^red^S^WMti^^ the words rather than their meaning- 
such tests were also culled. ^ , 

There are many possiblq facets. For example, within arithmetic — 
or even within addition alone — the number of digiti per number is 
a facet. The number of numbers to be added is also a facet. Thus, 
additbn by Itself is an inflnite domain; one can construct an infinite 
variety of addition tests which vary among themselves on at least two 
facets (the number of addends and the number of .digits per addend, 

respectively ) . ^ 

Not all of such further facets could be held constant because exist- 
ing batteries have not been constructed by taking jhesc systematically 
into account. Nevertheless./^ uided by the facet^culling strategy, 1 had 
arrived at some pictures for portions of the Thurstone batteries. Seven- 
teen tests from the Primary Menial A hiliiies monograph seemed to fit 
the culling rules rather well; their intercorrelations are given here in 
Table 2, twenty-one tests were selected similarly from the Factorial 
Studies of intelligence monograph; thcp'intercorrelations are in 
Table 1. 

My hand analyses of these tables revealed certain Interestmg cir^ 
cumplexes, taking two languages of communication at a lime against 
the' achievement-analysis fapet. This essentially duplicated, verified, 
and expanded the findings previously obtained from Guilford's data 
in reference 4. It was no^^ar how to unite all these interrelated 
circumplexes into one picture. The total picture appeared, multi- 
dimensional and too difficult to handle by my primitive (but effective 
for oiie_or two dimen.sions wBcn there is little error) hand.manipula- 
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tions* It seenwd.Best to show you just the subanalyses — i 
being arranged differently from the present Tables 1 and 2^ — and 
leave open the qu^tion of the over-all picture, 
. ^^ubsequin^ 1 and i were fed4nto the University of Michi- 

gafl "^COinputer with the new ndnmetric smallest-space-analysjs pro- 
gram, and diagrams were drawn from abtual Euclidean co§|dinatei 
thit reiulted from the calculations. One big surprise w^s that only 
two dimensions sufficed to describe each of the configurations, pThe 
nonmetric technique in general will yield a smaller spape than will 
conventional factor analysis; even so, two dimensions wire less than 
had been anticipated from the previous attempt to organi^i these data, 

' A second surprise was that, for the firstlimCj we had produced actual 
empirical pictures of what I have called a racto! 

Now the concept of the radex was introduced over ten years ago 
( I ) . It didn*t catch hold for several reasons, one of which was the 
lack of a good computing technique for plotting a radex,, Many empiri- 

^ cal examples^f parts of radexeSj especially those called simplexes and 
circumplexes, have been published, A simplex is a simple ordeyng of 
tests* or of any other s%t of interrelated variables. Consider, for exam- 
ple: addition, subtraction* inultiplication and division, Holding con- 
stant the facet of the number of digits involved, there is evidence that 

. addition, correlates more with subtraction than with multiplication, 
and least with division; two tests which are closer in their a priori 
order also correlate higher with each other. In a circumplex, there's 
a circular ordering of the tests; tests correlate more closely with their 
immediate neighbors and less with more distant tests around the circle. 
In the radex, ^both orderings happen si^multaneously. Tests are in a 
two, or -even higher, dimensionaj space, radiating out from some 
center. But still the contiguity relation remains. Tests closer together 
in the space intercorreiate more highly, and tests farther apart inter- 
correlate less. The computing problem \%: can we find an analytical 
way of plotting these relations of nearness and distance? This problem 

. has been solved, and the resulting technique applies to a wide variety 
pt problemSj not just to radexes nor just to correlation matrices. It is 
appropriate for a great variety of psychometric probiems in which 
only qualitative notions of distance occur, as weir as to data given 
originally in some metric form. ^ 
" We can see immediately no^ a great viftue of this nonmetric 
approach: you can compare your original data directly with the final 
diagram without looking at the intermediate calculations. You do not 
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FlGURE^ 1 



Data from Thurstone ^nd Thur||bne 
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^lave to know how you gi^t fi|pm one to the other;:the gpod iess of fit ' 
iban be checked directry arid visually. _ s ^ ' ' 

. i^^ Figures I and 2 are the graphic equivaients of the large.yorrelatioh 
g^ ^lgftbrnatfiGes borrowed frotti^e Thurstones, namely, TaWfcs i and % 
' '^'^\ : ^respectjvfeiy. I hope you vi|li^ agree that thlse are^ the simplest faithful 
' -portraykls yet made of suc^ lafge cmpiricar malices, along with those 
bf ratefaneei 4 and 5. T^oftests tenfl^to^be closer in a figure as 
their torrelations,.tend to tee Jarger in cb(responding table. The 
4ank clmlaU^ distances ^nd table correlations is 

\ ia^out^ S in each casei indic^hg quite satisfactory fits to the two 
f !|ub^i}iatHces. W 
..^lJiu^:pl©^t4s^s cla^^ as being nunierical, verbal or geo^ 

" ^^'ffifetrlcaMare symboUzed by ^W' ''V " or ^*G*' ftsp^ctively. Analytical 
*v.^i^ti..atd^lhdlcatcd b the achic^ft&t^sts are un^t^rred. 
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Figure 2 



Data from Thurstone 
Primary Mental Abilities (Table 2) 




N ^ nurnerical G ^ geometrie V = verbal '* ^ analytieai unstafred — achioveftieni 



In one battery, there were not enough Lmalyiical tests, I took tCfKts of 
achievement which were a priori more complex than other tests of 
their same lEnguage and classincd them with the analytical tests to 
build up more points in that region. This assumed complexity should 
behave similarly to analyticity in the present context, and raises defini- 
tional and other, problem^' which require deeper eonsjderation at a 
later date. , ' . \ f ^ ^ il^' ^ 

Empirically now, the computer gave the plots of Figures* 1 and 2. 
The relative distances between the pqints are as they eame Yrom the 
computer, I added the circular contciuA by' considering the dcHnilional 
system,. The blind empirical regults of the computer cann5t by them- 
selves show a substantive lavv of formation; they give only dimension- 
ality artd'^istance. Wi^^out the facetpd definitional systqm, it would 
be v^ry difflcult to interpret a plot, ' ; ^ 
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One of the most striking things about ihe^c plots, again to my, sur- 
prise, is.tlmt all the analytical tests appear in the middle. In each dia= 
gram the starred tests group around the center, and the achievement 
tests go out tpwards the periphery. 

When first discussing' the rad^x for intelligence tests some dozen 
years ago, . I hypothesrEed it would express a radial expansion of 
complexity; simplicity would be in the center and expand outwardly 
into complexity. Complex tests in different areas would tend to be less 
correlated with each other because Jhey go off in different directions ^ 
of complexity. The present data show that quite the opposite may be 
triie, althou^ complexity is not the same thing as analyticity (the dis- 
tinction between being complex or simple is not the same as the'dis- 
tinction between analysis and achiet^ement). In Tables 1 and 2, it is 
the analytical tests that tend to correlate more with each other, as is 
shown by their greater niutual pro^cimities in Figures 1 and 2, 

A second striking thing about these plots is that the plane seems 
to divide into three distinct sectors, one for each languagt; of com- 
munication. In Figure 1, the verbal tests (the V^s) all fall in the upper 
region, the geometrical tests (the G^s) fall lower to the right, and the 
numerical tests (the N's) fall towards the left half of the plane. Excep- 
tions are three G tests, coded as ^^G(N)," which fall into the numeriy 
cal region. These arc all tests of cioi'Countmg, a combination of 
numerical and geometrical concepts. They use dots, and hence are 
geometrical by language of communication; but the exhibited rule — 
counting — is numerical, 

A similar exception is Test 5. coded "N(V)'^ it gives arithmetical 
problems stated in verbal form. Considering our a priori culling rules 
for the present study, we had debaied whether or not to leave it out, 
because of possible confusion on language of communication; some 
numbers, of course, occur in the midst of all the words. However, 
Test 5 was retained; it turns out to fall into the verbal region of 
Figure 1. It perhaps is better classifiecI.MS y(N), since its language is 
most predominantly verbal, despite fact that the exhibited rule is 
arithmetical. It is interesting that tlfie dot^counting tests fell more into 
a region corresponding to their r«fe. rather than their language, while 
the verbaUarithmetic test behaved rnqstly according to its language. 
The same general kind of radex appears in Figure 2 as in Figure 1 . 
The verbal tests are off in one region; the geometrical tests tend to be 
in another region; and the numerical tests tend to be in a third region. 
These figures arc from two different batteries. These are different 

page 34 _ 42 _ 



Louii Gyttman 



sets of tests, given at different times and to diflferenj populations' 
Figure 1 is for eighth grade children in Chicago and Figure 2 Is for 
university students in Chicago. The correlations of Table 1 were caU 
culated directly as product-moinent correlations; the correlations in 
Table 2 are tetrachoric c^fficients. Sampling errors are also Involved 
here. But the over-all picture In both cases is the same. Similar results 
can now be seen by looking agiin at the nonmetric analyses I had 
made of dto of Guilford (4) and Coombs_(5) before the computer 
fechnique was available, and^before* tf^ present radex became 
"obvious.*' The next step Is to study even larger batteries, allowing 
more facets to va^ systernaticallyj and to sec if the multidimensional 
structure pcedidted by the corresponding facet design does in fact 
occur. « 

What are the basic implications of this kind of structural analysis 
of interrelations? First of all, for those who have been interested in 
factor analysis, this may throw a different light on the problem. The 
purposa'W factor analysis has basically been to study configurations 
of points. It just so happens that it is difficult to do this without cal- 
culating coordinatd systems, but perhaps we have placed too much 
emphasis on such coordinate systems. The emphasis should be on the 
conflguration itself and not on the coordinates. By emphasizing the 
coordinate systems, we have been led to try to call them '"factors" and 
■to n%me them. Considering diagrams such as Figures 1 and 2, I think 
we^ can realize that perhaps one should not look at. concepts Ijke 
"numerical/* --geometricaU" "verbal," and so forth, as nar^ies of 
factors, but .rather as elements of rules for itim construciion, Wa can 
give detailed instructions to Item constructors on how to make up types 
of items we want by using these and elements of other facets implied 
in the definition of intelligence above. We are exploring and using such 
facet designs in our test construction program in Israel 

A second important implication is for the problem of prediction. 
If there are externar criteria we wish to predict ~ siich as success in 
college or success on the job — having a simple map such as in Fig- 
ures 1 and 2 enables us to deduce what the most parsimonious way 
may be for accomplishing this. The ideal thing would be to have a 
map of the set of criteria as well as of the predictors and to see what 
; the joint map Is. The combined space will usually be larger than that 
for each set alone, since the criteria and the predictors will differ 
usually on some facet which Is constant within each set separately. 

That point in the predictor space closest to a criterion in the com- 
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bined space can be called the image of that criterion. In general, a 
good way of predicting a criterion is to use the tests which most 
closely sunrourid its image in the predictor space. For example, if a 
criterion's image happeni to be amidst the four points in the lower 
ri^t hand corner of Tiguri 1, then a good choice of a small set of pre- 
dictors is*tiie corresponding four tests (namely, tests 18, 3U 44, 38). 
Adding more tests from outside this region generally won't help the 
prediction much. Perhaps this helps explain why, in the past, people 
have kept reporting it doesn't pay to use more than three or four 
variables in, a muhiple regression. Adding further variables adds 
largely to sampling error'and decreases cross-validity. But should the 
prediction battery be not much more thali two-dimensional, this would 
help explain why there is no point in using more than three or four 
variables as predictors* Knowing where a criterion's image is in the 
predictor space will tell how best to pick an optimal subset of three 
or four predictors. Each criterion will, of course, have its own image, 
and hence its own particular optimal sub-set of predictors, 
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Psyohometria 
Approaohes to 
Faator Analysis^ 



Henry P. Kaiser ^ 
The University of Wisconsin 

The title ©f this paper is seemingly delimiting: apparently 1 am only 
to talk about the "psychometric'- side of factor analyBis. This restric- 
tion gives '^substantial pause for thought, for what indeed does '*psycho- 
metric*' mean? Is it unduly restrictive? 

First, the term can be used in a negative sense, so that I am not to 
be concerned with recent developments on' the statistical (inferential) 
side of factor analysis; consqqucrttly, I shall not talk about sampling 
distributions, tests ^f signijicance, and all the rest of the jargon asso- 
ciated with t^ problem of malfing probabilistic inferences from sam- 
ples of individuals to populations of individuals; we shall consider 
that populations of individuals are available — or, more realistically 
~ that our sample-populg.tion inferences are non-probabillstlc. There- 
fore, let us first pnsider the psychometric side of factor analysis as a 
concern for what indeed is the mathematical model of factor analysis* 
arid I^w, alge^aically, one thinks about it. My predecessor on this 
program, Professor Guttman, once said informally, **Wc have to clean 
up the algebra of factor analysis before we can, appropriately, hand 
It over to the statisticians." My successor on this program, Professor 
Tiicker, has rriany times said essentially the same thing. Since these 
two gentlemen are undoubtedly the two heaviest -of the heavyweights 
in the theory of factor analysis, restricting consideration to the non- 
statistical side Is, in all likelihood, not entirely a sterile enterprise, 

TYm tenn **psychometric," though, has an important, and some- 
what more pretentious, meaning — with connotations that are dis- 

* This paper was pr§pafed while the author wjp the L. L, Thurstone Distin- 
guished Fellow, Psychometfig Laboratory, Unrversity of North Carolina. 

i 
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tinctly of a scientlflc. Inferential nature. Often hidden in the maze of 
algebra and computations of factor analysis is the fact that these pro=' 
cedures are ultimately rather explicitly concerned with inferences 
about domains of scientific content — i,e,, that we arc faced with the 
important problem, beyond merely statistical description, of attempt- 
ing to infer the nature of the structure of a large — usually arbitrarily 
large — battery or universe or domain of variables or tests on the 
basis of a limited %Bmp\e iselecthpn might be a better word) of vari= 
ables. In this sense, factor analysis might be called ^^psychometric 
inferential/' and an explicit concern for this kind oF^cieniific inference 
is indeed the first problem of factor analysis, ^ 

Before I get down to brass tacks, allow me to delimit myself in a 
way not suggested by my title. I shall restrict myself to considerations 
of what might be called traditional — or exploratory — factor analysis. 
I can illustrate this by giving two extreme kinds of situations in which 
factor analysis is applied. Often, a consultant is confronted with a 
seemingly sincere **investigator** who presents a large table of num- 
bers and wants to a factor analysis, and 'for which the apparent 
principal justification for proposing to do this deed is that the large 
table just referred to is neatly written out. At the other extreme is the 
proposed factor-analytic investigation for which every variable to be 
included has been subjected to the most searching rationalization and, 
with this case, the hope h that very specific hypotheses can be adjudi- 
cated definitively. The former extreme is, of course, to be eschewed, 
while for the latter, there are perhaps more powerful techniques avail- 
able (some of which are factor=analytic in nature, thanks to Professors 
Tucker and Guttman), ITius, I would like to put in I plug for steering 
a middle course and suggtJst that fact^malysis as it is usuaHy known 
enjoys its greatest justification^as an ^loratory technique, by which 
the variables under consideration all^enjoy a reasonably weU ration- 
aiized, but not necessarily certain, probability of belongirtg to thb sci- 
entific domain of interest, ^nd tlie structure of which is essentially 
unknown, . ^ . 

From this viewpoint, factor analysis is an unpretentious *^bringcr of 
preliminary order out of ^elUperceived chaos;' a technique which 
is capable of generating ideas rather than providing final answers. 
Thus, be reasonably — not compulsively — careful in planning a fac- 
tor-analytic investigation. But don^t take your results with such 
seriousness as to preclude further — probably more penetrating — 
analysis. 
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It is convenient to divide the theoretical problems of factor analysis 
into two groups: (a) those concerned with finding an interesting fac- 
tor spaca, the so-called communality problem, and (b) those con- 
cerned with finding an interesting basis (coordinate system) for the 
interesting factor space, the so-called rotation problem. It is almost 
accurate to say that these two problems, conceptually, are independent 
of one another. First, then, the communality problem, ' 
. .D, N. Lawley- in 1940 gave what we now recognize as one defini= 
tive solution to the problem of factoring. This, is his famous maximum 
likelihood solution. Subsequently, much work has been done with his 
procedures, particularly by Lawley himself, and by Rao, who, in 
clarifying and specializing the method, renamed it "canonical factor 
analysis." It is perhaps best described in Rao's terms as the solution 
of a certain problem in canonical correlation, and operationally as a 
principal axes solution of a rescakd, reduced correlation matrix. That 
is, the correlation matrix, with communalities in the diagonal, then 
ha^ ti\e variances of the variables changed in a way to solve a ^oblem 
in caftionical correlation. The result of this rescaling, or change of 
metric, has the most important ''property of leading to solutions which 
are invariant under any rescaling: the Holution is scale-free, different 
from many traditional factoring rnethods — e.g., that anachronism, 
the centroid method. 

Now the Lawley-Rao method, in particular, rcscales in the metric 
of the unique parts of the observable variables, but to obtain this pro- 
foundly important property of securing scale-free solutions, it is sufii- 
cient (but not necessary) to rescale in the metric of the unique parts; 
one can go to the other extreme and rescale in the metric of the com- 
mon, or communality, parts of the observable variables, John Caffrey 
and I noticed this a couple of years ago; more accurately, we noticed 
nothing more than that one could write down an eigenequation par- 
alleling Rao's — and wondered if it meant anything. Apparently it 
does, for in groping for a rationafization for our writing Rao with back- 
wards rescaling, we found that we were, in solving this affair, deter- 
mining common factors which successively have maximum reliability 
in the generalized Kuder-Richardson sense — i.e., factors with maxi- 
mum "alpha,*' to use Cronbach's term. 

For those of you who are interested in more detail in these two 
scale-free procedures for finding arbitrary factors, let me refer you to 
our forthcoming paper, "Alpha Factor Analysis," in PsychometrikiL 
I rike to think that this paper is quite readable; I would be the first to 
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admit that our very psychometrically oriented procddure undoubtedly 
enioys its prineibal justification today because it was provoked by 
attempting to taitate the work of Lawley and Rao. I wou^Jhope that 
those who read our paper will b.e ablr to provide Caffrey and me with 
more incisive understanding of what is going on. 

Both attacks described above for the communality .problem were 
developed most fundamentally under what might be called the "pure 
Jactor^analytic model o^L, L. Thurstone. As many are aware, there 
are other models, factor-analytic in flavor, glvmg factor-analytic-likc 
results in practice, but which arc not strictly factor analysis. I refer 
here to the monumental work of Hotelling in his component analysis, 
and to the monumental work o;t Guttman in his Image analysis. My 
reference above to the problem of finding an "interesting factor spac^ 
rather than a "common factfa'r spflcc*' wa^ deliberate; "I most certainly 
do want to talk about both component analysis and image analysis, 
because practically these methods will yield results which differ only 
mihutely from those obtained .under the pure factor-analytic model, 
while they enjoy qiany elegant' technical simplifications, and in prac- 
tice have Ihe tremendous' advantage of being capable simply of solving 
the factor: score problem (in "pure" factor analysis, the factor score 
problem is insolvable* ). * ' 

■ 1 do not have the tijric to describe Guttman's image analysis; I will 
only assert — and refuse to stand for any arguments — that his basic 
1953 P'^ychometrika^papst is Tcquircd reading for the serious student, 
for all who aspire td- be lifted out of the primordial. mire which pre- 
ceded it Now, Professor Outtmah did not explicitly develop his ideas 
with direct reference to work-a-day factor analysis; .this link wjis pro- 
vided by our chairman^#^ssor rtarris Jn the most important paper 
to appear in the lastjffin yeirs on the communality problem.. It is 
Psyc/iomemA-a, 1 962'aadfirJ^sd required rea 

What Harris accoiflpligiid'. was to bring.togcthcr many of the. pro- 
vocative psychometr|f 'pt#ertius of image analysis and show their 
intimate relationship to R^'s\ ;version of maximum likelihood factor 
analysis — agaih Rrdbabl||t53tfbr termed "canonical factor analysis. 
The crucial . insy« httrci^i bbught about again by rcscaling: in 
order to oBtfti IsbaWrcfl soUHion, Harrb rescalcd in the metric ot 
the anti-lnii^s:(thc atiljlOfi|>£ unique pi^n &.ctor analysis) and 
nrov|d||;i|;ri|ht |wni||tine,;with the mtt#=analytic vcrs«,n ot 

^S£^ilurat|yi?^l^i|lutipn 'Upes not exist. 
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canonical factor analysis. Most important practically^ in this context, 
Harris solved the **number of f^ors" question, but I shall dbfer talk- 
ing about this until I summarize ffle communality problem beldb 

The tbree solutions described above* 1 T Rao-Lawley oanonic^l 
factor analysis^ 2. Kaiser-Caffrey alpha factor analysis; and 3, 
Guttman-Harris image, analysis^ let me reiterate, all haMenhe elegant 
property of yielding factors which are invariant under cjfanges of*scale 
—one no longer need be concerned about units of measurement, as 
the same results are obtained regardless, Initial factoring procedures 
which do not have this property deserve only one treatinent! the ash 
can. What then about the popular use of Hotelling's principal com- 
ponent analysis^ which does not have this property? The rule of thumb 
of taking only such components of the correlation matrix with latent 
roots or eigenvalues greater than one — proposed years ag6 by me 
originally for no really good theoretical reason — has disdainfully been 
referred to as '*Little Jiffy'* by some of my thoughtful colleagues. It 
turns out though, in a component analysis setting (no unique fac- 
tors postulated), that if we maximize Cronbach's coefficient alpha 
rather^than Hotelling's variance accounted for. Little Jiffy can be res- 
cued from the ash can of non-scale-free soluUons; more .exactly^ if 
we maximize alpha, we automatically rescale into the correlation 
matrix associated with any covariance matrix and thus, arc able to 
determine scale-free components. So, ^more or less after fhe fact, we 
can add a fourth scale-free prqeedure ~ the popular Little Jiffy — 
for determining an interesting factor space. 

These four methods can be summarized in the following two-by-two 
table: 



Canonical 


Image 


Factor 


Analysis 


Analysis 


(a la Harris) 


Alpha 


• " Principal 


Factor 


Components 


Analysis 


^ (Li|tle Jiffy) 



'4f-, AIJ four procedures involve principal axe^ solutions, with rescaling; 

11'^ all four are scale-free. The two on the left are "pure"' factor analysis 
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and may be obtained operationally by iterating on Ihe solution imme- 
diateiv to its right. The two on the right in this sense are first approxi- 
mations to those on the left, and while they are not factor analysis in 
the strict sense, they do yield very similar results to their counterparts. 
They have the substantial advantage of giving factors which are linear 
combinations of the observable variables and thus, of yielding scores 
on the derived variables or "factors" with no fuss — in stark contrast 
to the absence of a unique solution for factor scores in the purely 
factor-analytic solutions. 

Probably the most important side of the communality problem in 
practice is the hoa^ old question of the "number of factors," In the 
foregoing two-by-two table, the, number of factors which come out 
"naturally" in the first row, for canonical factor analysis apd fer image 
analysis i la Harris, is Outtman's classic strongest lower bound, ern- 
pirically "top inany" ^ about half the number of variables observed. 
For the second row ^ alpha factor analysis and principal component 
analysis of the c6rrelation matrix (Little Jiffy) — the number o fac- 
tors which come out "naturally" is Outtman's classic weaker lower 
bound, the number of eigenvalues greater than one of the correlation . 
matrix, usually about the "right" number, 

I might say that Outtman's weaker lower bound in giving about 
' the "right" number, can sometimes give too few, and underfactonng 
is a catastrophe. His stronger lower bound, which mvariably gives too 
many can never lead to this disaster of undcrfactoring; overfactoring 
is usually only a mild irritant, which may be cured by careful thought. 
I wish that I were able more successfully to promote this idea, but 
■ generally practitioners want exactly the "right" answer for the num- 
ber of factors to be given without careful thought, and that, gentle 
readers, is as unattainable as Eldorado, . 
''I'm not going to reveal which of the four solutions discussed above 
is, the right one: the question really is without meaning, I will say that 
four is a somewhat smaller number than the myriad of techniques one 
is confused by in the earlier literature. 

The result is an interesting factor space, preferably found by one 
of the four methods just reviewed. Its basis, however, is not inter- 
esting—being some variety of (rescalod) principal axes ~ so this 
original basis should be changed by a linear transformation — i.e„ it 
should be "rotated," What is the current state of this general problem? 
For objectivity's sake, I shall restrict myself to analytic methods; con- 
sistent with considering fajtor analysis in its traditional exploratory 
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vein, I shall also consider that I am going in "blinds" searching for^ 
a structure, ratjier than testing ^for the existence of a postulatedr 
structure, ' " ^ 

In this venture, I first distinguish the so-called ''oHhogonal" from 
the so-called ^'oblique" case, according tp the kind of restrictions I 
place on the transformation matrix. This distinction usually — but not 
always ^ — bears a one-tOKjne relation to whether I restrict the trans- 
formed factors to being uncorrelated or not, . 

In the orthogonal case, present criteriii are almost invariably of 
the form: 

Q -- wK maximum 



•where Q is the quartimax function, and K is the function, which in 
conjunction with a weight w of one turns the criterion into varimax. 
More generally this weight w controls the variability in the VElriance 
contributions (column sums of squares) of the factors; at one extreme, 
for w = H ooVwe have principal factors; for vv^ = 0, quartimax; for 
= 1, varimax; for w = one-half the number of factors, Saunders' 
"equamax;'* and for w =^so, all factors have exactly equal column 
sums of squares^Recently, 1 have looked at this class of criteria with 

%ome cafe, particularly at values of w greater than one, in tne hope 
of obtaining orthogonally rotated factors with more nearly equal vari= 
ance contributions — a hope generated by the concern of some inves- 
tigators who have expressed little doubts here and there a^s to the 
ubiquity of varimax. Unfortunately, 1 have been Sble to proye that it 
is always possible, for any trans-varimax solution where w is chosen 
greater than one, even by a smidgen, to produce examples/ in which 
utter catastrophe occurs and two pure independent uncorre/ated clus- 
ters will be subjected to a 45 ■ rotation, I have also looked a^ criteria in 
this class where w is generated by the data; here I have been able 
to prove that the factorial invariance (in simple cases) wjiich obtains 
for varimax cannot, in general, occur. I must conclude^ that within 
this class of solutions, varimax best docs the deed. But^. here "best" 
has mostly a negative connotation: the old reliable work-horse vari- 
max wilt never result in catastrophe, while other criteria in. this^ class, 
perhaps better in specific problems, can be disastrous in others. 

It appears, then, that further p^ogre^s in the ortho|dnal case must 
Gome from an entirely fresh start. The most promising development 
K|at I know of occurs tangentially in Peter SchonemAnn's recent doc- 
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toral dissertation at^e University of IlHnofa. He may have something^ 
very good for us in the near future. * 

Analyticaily, in the ^'pure" sense of writing down some to-be^ ' 
optimized function of the loadings to be obtained via a non^orthogonal 
'^i^fprmation, the oblique case has been quiet for about five years. 
' Wst serious effort in this line of thdugfit initiated by John Carroll 
/arl3 followed up by Dave Saunders, Kfef n Dickman and me was some- 
thing dubbed :*'binormamin'' by Professor Dickman and presented by 
two of us at ^e '59 apa meet^gs. We discarded it, more or less, ^ 
Sibecause of the simply awful cprfiputational problems in\^olved and 
' because of its apparent sensitivity to the number of factors Retained 
^^(it would seem to colldpldthe factor space if you gave it a dirfy look), 
^ There has been a minor revival of interest now that we have the super- 
speedy computers to do the |ob; it appeals that some of our initial 
concern was overly pessimistic. The one positive statement I can make 
about it is that when it works, it works fine — better, say, than the 
earlier oblimax and oblimin crite^/ IA general, though, any analytic 
, oblique criterion is not to be trusted routinely to give adequate results 
'' because of the apparent lack of bur ability to apply the cdnstraint of 
non-singularity on'Uh^ transformation matrix in t^e':jtra,ditiona! ap- 
proach to obllgUeV'rOtation,'' 

The terrible terroi; ^f singular transformations in the oblique case 
was resolved in the swriimer of '63 when Profes^r^ilftris suggested to 
me that we apply orthogonal transformations to;'^cK^erateiy rescaled 
and then ^^mis-scaled" principal axes sojt|itionH. Paifting him gent^>|r 
the head, I pointed out that this was of course illegal Smiling sertfhdy, 
he decimated my conservatism by pointing out that after such trans- 
formations, all the preliminary scaling can be taken out very easily 
with non-singular diagonal matrices. The result? All possible trans= 
formations, and thus all possible factor analytic solutions — involving 
uncorrelated or correlated factors ~ for a given factor space can be 
obtained with orthogonal transformations. This is a Big Breakthrough, 
for being able to reduce the, entire rotation problem simply to one of 
applying orthogonal transformations makes the problem really trac- 
table for the first time. Already Harris has brought this.entlrcly new 
framework to bear in solving definitively the problem of cluster analy- 
sis. Further applications will undoubtedly be forthcoming soon. Be> 
prepared: read ^^Oblique factor analytic solutions by orthogonal trans- 
formations," soon to appear in Psychometrika, 
^ Doctor Schonemann is now at tte University of North Carolina. 
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I shall stop now. Let close on* an optimistic note^^^on't think 
rni becoming lenile ^hert I say that, psychometrically, ^fe period of 
the algebraic explication of traditional exploratory factor anajygis _i3 
drawing to a close. We are uear to having things well enough in hand 
to give th^ model to the statisticians. Perhaps pretentiously, I think! I. 
can say that we psychometricians are like Galton, who, when haying 
the correlation coefficient Sufficiently under control cpnceptuailyv' 
turned the thing over to Karl^arsp'n/,- ■ 
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Multi-mode f^^or analysis^ is being dcveioped to help meet the naeds , 
for a procedure tQ' search for and to represent relations existing in 
data from observaiiflns #iich may t>e tlassifled according to several 
identifying/attributes: Initial discussions of this development appear 
in the monographs: Probfems m Measuring C/if//ig^ edited by Chester 
Harris W^md Cpntributiohs to Mathemgticgl P^ychahgy edited by 
Nonnan Frederiksen and Harold Gulliksej} (9)1^ , 

An example of the simplest casS fdr an identifying- attribute of 

-observations. would be when a^single test is administered to .a group 
of itidividuals/The scores* onAhi^ test constitute our abseryations and 

leaQth^bservation is identifiable by the individual Thus, the Individ- 
9&stltutG flrt,.i^ntif^ these observations, of ; 

"i tndde.Qr An alternative example is when 

Mingle-l^lVWual' ik;;^^ large number of tests,. Thcn each score 

' -can tec identified ;% tiifi;test. The group of tqst^;Jrt(^- constitutes a 
mode of identifying classitication; A;;MMP^up in complexity of classic 

■ ficatlori ^occurs #en. say, a group of ^^pple is givcji a group of tests. 

■ Itow ea^:h scorils identified by the person and by the test, thus involv- 
ing twcirh^des of identifying classilkiAion. These data may ba re- 

' corded in^ a:tablc with a row for^each individual and a column for 
each testrfiiich score is record^diimone^ge of such a table. Tradi- 
tidnal factor;yaps;^ represent rela- 

^ tions in sucjl d^tfii . . , 

■ A next s'te^^up^ard in compLQjity occurs when, for cxampl^, a 

^vTbe ruHcurch reported in ihls ^papcp was Jointly supported by -the; Unlycrsity 
Illinois u«^c om& of Naval Research umler contract Nonr 183^ (39). 
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^:.C:^-frq5U|)p^ baUery qt tests ori a lumber of different 

v^-^ '^^iSbcasto be identified by the jndU 

V 'vidual ^ 1^^ ^ t h a§ r nn^t i tnti n g three^ piodes of 

■ identi^g dassmcarions. These data may be thought ofjas being 
; > i^cQrded ia a threc-dimensionaL table such as 1^ data boxi; illuS' 
trated in Figiife 1 where thert is a row in iuode; I- for each occasion, 
a colunin in mode 2 for each test, and a-kyer in mode 3 for each 
. individual. Each cubicle of this box contains one test s^re? Multi- 

. mpde fa^^^F analysis is bein^evised to trea#data such as thpse in 
this" dat^^ ■ ' ■ . ■ ^ " ~y , 

An ext9nsi^e variety of situations can be conceited which involve 
' . . three oy^lfribre modes of identi^ing classlticatiqps^ First is the example 
just discussed. A second extople occurs when a |yumber of indi- 
^ viduals aro ratfed on"a number of traits by a number Wfolher individ- 
uals; an analysis of such data Will be presented. Ah ft\ternative to the 
preceding example is when each of a number of inAviduals is rat^ 
on a number of personality traits in different kinds of .sftiUlions.^n 
extended example would involve four modes: persons, traits, occa- 
sions, and situations* In this example, occasions might be interpreted 
. as particular periods of life sOch as enteriiig first grade, in the middle 
of first grade, toward the end of first grade, 'btc^ the situations miiht 
be: in the classroom, on the play yard, at hcHneJ etc. Such a study 
migl^t have extended relevance to developmental psychology. A third 
' ex^imple could be when individuals arc learning a conjplcx task on- 
which a number of different scores can be 'taken on a numbe^of trials, 
-Such data constitute oijr second example to be presented. The third 
example of analysis to be presenfed involves individuals imagining 
that they- are in a number of difl^int situations and indicating the 
extent of ^^esponse for diferent modcH of ^responding while in these 
^ situations, „ ^ ; ^ 

' Figure 2 presents a scljcmatic rcpre^falion of the ;4h!^c/mode 
factor model. There is a /actor matrix foV each mode of identifyii^ 
classification, mode 1 on t^ left, mode 2 at the righl.|^d modir^ 
as if in depth. Each of these, factorf^iatriqcs relates ^ observed 
variables of the mode, (if ideitHying classification or factor variables 
f fof the mode. Thus:, if method of rating c^tiiutcs mode I, the rows 
, oflhe mode 1 factor matrix would be the various methods of rating, 
and the columns would be conceptual methodH of rating. The observed 
methods would be described in terms of their relation to- these con- 
cepttial "methods of rating^Voj^ the three examples to bl^ described, 
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:pj4e 3 be the individuals on whom the obier-; 

vaUons ^re mate io'tfc^ columhi of the mode 3 factdj^ matrix in 
— .^^yfe^ The jQwi this matrix 

rtp^enteoneeptualiEe 

: jrC^^^^ the procedures describe in-thi 

^no^aph i^i^ Mathematical Psychology (9) is a pro- 

cure by whieh allowBnc^ can be made fo/ errors o£ measuremeftt ^ 
-and ofter infliieflc^i that affect the measures far each particular com- < 
bination bt variables in mode 1 and mode 2^ raises a problen^ 
^^^r^^aai^ffl^l^^tte^TOa^^ tradftional factor analj^iis^ 

arid results in an indetermlnancy of the entries in the mode 3 factor / 
' mttrix^' This is analagous to the facto^ score problem. The thr^^ factor ^ 'S 
i^ttiM are tied tdgether by a iniall core box which gives the rela- ^* ; ; 
tibni among tiie three types of idealized entities. A poipt of interest^ , 
is that^Jlw new procedure utilizes a matrix of mean cross products 
bihv^iPl^Mables anri to the Campbell and Fis^ multitrai^ . 
i&lrimethod matrix (1 ) . The examples will present the niode 1 and 
v.mede 2 fkctor matrices and the core hpi for each case. , ^ ^ ^ 
*Gonsider Table 1, This presents the results of an analysis of multi- ^ 
trak^lUmethod^ d^^ These data wdre cpllected by Professors^ ^ 
Xoweirkfeily and Don^^^^^ (5) as part of a research project ort"" : 
the selection of clinical p^chologUts conducted at the Umversity Of \. 
Michigan, Data were supplied by Professor Donald Fiske. The, analy- ' 
m wJL^hducted by Mr. Edward Hoffman (4) while he ^as a gradu- 
ate student at the University of Illinois/ The methods of rating con- ; 
stitute mode 1;, these methods are ratings by the staff, by the ieanfi- 
mates, and by the trainee himself. The mode 1 factor matrfic is gwn 
the u^er left. Two factors were extracted, a genera! lactor and^ 
factor related p the sell-rfltings. These. fetors are unrotated principal 
axes. Note that the staff- and teammat^nobserved mcyiods of rating 
are vei^ similar whereas the self-rating depends in^art on the general 
and in part' on*the specific idealized rating method for the self. 

Fiye factors ^ere found for mode. 2 having to do with the different 
traits thai were rated. Fifteen traits w«re splect^ed from the 22 that 
actually were, rated. These 15 were to represent the five recurring fac- 
tors identifted by Fiske (3), The firs; factor strongly apparent was 
the social Adaptability factor, the secohd was the factor seriousness or 
CMScientidusness, the third trait factor was an inquiring intellect, 
whereas the fourth appeared to be epotiona! control. The fifth factor 
was ley^ell defined, but appears to be rejated to self-assertiyeness^ 
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19S4 JnvlUitlonal Cdiiftr,«n€# dtt Ttitinf Problemt ^ 

Eatriei for the core box are^ given In the lower middle section of 
ttie table. The first person factor is characterized by high values for 
jtf^iffi ^adaptability and se ^f-assert^v as rated on the general rating 
method factor. A person who was describable as high on the first 
person factor and near average on the remaining person factors would 
be fated highly social adaptable and highly self-assertive. All other 
ratinp would be near average. Another person characterized by^a 
low vdue in the first person factor would tend to be low in social 
adaptability and low in self-assertiveness when rated by the general 
rating factor. All self-ratings would be near averap.^he second per- 
son factor is characterized by a high positive loading on seriousness 
and conscientiousness and a high negative loading on self-asiertive- 
ness when rated by the general rating factor, The third person factor 
also Involves the general rating fact5r and positive values fbV an 
inquiring intellect and self-assertiveness. The fourth ^rson factor 
brings in the self-rating method factor to a greater extent vi4iere the 
individuals high On this factor rated themselves high on inquiring in» 
tellect and emotional controL The fifth and sixth person factors involve 
the emotional control as rated on the general rating factor with dif- 
ferent combinations of seriousness-conscientiousness and self-asser= 
tiveness. It is of considerable interest that the self-rating factor is 
involved to a marked extent only in one of the person factors, factor 
four. This person factor may represent the difference between the way 
a person views himself and the way others view him. We may sum- 
marize the results, then, as indicating a fairly strong communality of 

, ratings by the different rating methods in the general factor with the 
self-ratings diverging in part only toward the self-rating factor/ The 
trait factors are relatively clear and identifiable. The person factors 
indicate an area within which the self-ratings diverge from the ratings 
by the staff and teammates. ^ 
Results for our second exampic are given in Table 2, The data for 

■ this analysis were taken from the Psycholngiual Monograph by James 
Parker and Edwin Fleishman on the Ahiliiy Fai iors and Component 
Performance Measures as Pre(lk torsj)f Complex Tracking Behavior 
(7). The tracking device employed wuh^ constructed so as to simulate 
roughly the disphiy characteristics and control requirements of an 
airborne radar intereopt mission. Time of the tracking performance 
of each subject was divided into a number of trials. On each trial four 
measures were taken: a measure of horizontal error, a measure of 
vertical error? a sideslip error, and a time=on-target measure. Data for 
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ten of the trials were presented In the form of correlations in a matrix 
analagous to the CampbeU and Fis^e multitraltTmultimethod correla^- 
-HeB-^atrixryt coniiquenre of wing enrrela^^^ in this situaUon is 



that the an^ysis involves consistencies in deviations from the mean 
learning curve for eftch measure. . « 

Mode 1 was concerned with the spring measures. Again there were , 
two factors. This time one of the factbrs was related to direction errors 
involving the horizontal errors, the verUcal errors, and the tlme-on- 
target.lnrhe second measure factor was related to sideslip control m- 
vgl^ng the sideslip error and tlme^jn-target. These two factors mdi- f 
cate^^niarked distinction between the two aspects of control ui this 
trackinl'test. 

The mode 2 factor matrix Is . for .trials and appears at the upper 
right of Table 2. Four factors were apparent, one for very early learn- 
ing, second for middle early, a third for middle late, and. a fourth for 
very late trials. These trial factor^ constitute. standard patterns for dis- 
crepancies from the mean learning Curves, ' • 

The core box Is at the bottom jiohion of the table. There are seven 
person factors, representing seven dimensions of individual differences 
in-performance on this complex 4rackin| task. The first four person 
factors involve the direction error factor. The first person is for very 
early performance on direction errors, the second is somewhere be- 
tween middle early and middle late on direction errors, the third 
emphasizes the middle late direction errort, and the fourth factor tends 
to emphasize the very late direction errors factor. The fifth, sixth, and 
seventh factors are concerned mainly with the sideslip control- the 
fifth factor* involving very early performance on sidesiip control, the 
sixth factor being related mostly to middle early and middle late pcr.- 
formance on sideslip control, and the seventh factor being related to 
the very late performance in sideslip control. 

A major point of interest is the almost complete disassociation in the 
person factors of effect^ of the direction errors factor and the sideslip 
control factor. Only during the middle late trials factor docs there 
seem to be an interaction between, the direction errors and the side- 
slip in discrepancies from the mean learning curves for, the measures. 
This independence should be important, to our theoretical considcra- 
tions of such learning tasks and to our attempts to measure perform- 
ance on them. If similar independence of measure factors were ob- 
served for other learning tasks, serious doubt would exist as to the 
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equivalence of a number of learning experihients when different meas- 
ures of performance were employed in these experiments. 

The results in Table 3 are taken from an analysis by Dr, Joseph 
Levin (6) of situation versu|.mode-of-response rating data collected^ 
by Norman Endler, J. McV. riunt.'and A. J, Rosenstein (2). Mode I 
involves the mode of response. There are 14 difTercnt listed rjiodes of 
response given in the upper portion of the table. Eleven situations are 
shown In the middle section ofnfic tabic. The task of each subject was 
to imagine that he was in ^ch of these listed situations and to rate the 
. e?ctonUhat he^ would respond for ^h of the modes of response. The 
rating scales were from 1 to 7. Jhc ratings^ were standardipd oyer 
individuals for each mode of Tesponsc within each situation. . 
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19S4 Inyititloiial Cenftrsnsjl^d^:^ 

* 11a mode of rgspomiyiSmt^^ .toit^ given in the 

up^r portion of the tabl©; ,5^^ffa^^ frabd^^ llsted^rst 
have M pFWa^ffl^OT^faloy^ 



The next fbur modes of i^pi^^/^aye^j^^Ji^djif^ a factor that 
could be called Exhilarat^h^^^d !^ li^ii^M of response are 

grouped under a factor ttiat d^^ifbe*^ responses. 

ITie factor matrix for,sitiaiti^iVijr |^ 2 in the center 

of flit table. Five of the itl^t^^ 4en& Jo ^nder a factor that 

^^Mld be called Interp|rtdrtiS ^^u^ of the situations 

have tii^ loadings otf ;th^/$^^drsj^^ which might be 

called Inwumate iituatipri|/;^t^ highly on 

the third factor^ which pfi|^©t(^uS^ seemed to have 

a cotomon characteri^tic-ift^lhs^^ it were, going into 

a situation whose exaetjtoatii^^ 

He results in the cbr^.ttoi"^t)jfte jbotfi^ table are especially 
interesting: There^^eto fe' be'f^l^ which could be 

characterized as idfealj^e^^p^riotis^^ person seems tovreact 
with both General ^jstr^ss an^^^ all three types 

of Idealized situatidttsV^^;4p|atiy^^^ this persort factor would 

not be distressed l^bf 4^ve,s^ut^nok^i^> Mactions to . these idealized 
situatioh factors. Ptfirep4$^^4rit^ike^ fcither high ^ pqsiUve load- 
ings on the first ptersdh' f^^ loadings pa tliis flbt 
persojj f actor Would ^e aSput 4 v^a^e . to vE?(h 11 a ration r^sponsa^ to ^ . 
the oonceptuAi '^uatjpn idealirid person v^odd '! 
be characterised by3x^il&ratioi> t^'a^^^^^^^^ types gf situations, esp|^ 
'Cirilly ftie ^naniimaie;';pfl^'ii\^^ of 'Dis^rcis,to this InatSw^^ 
mate lituation^ :>^ilp: h^;^^ in AutorfdM|'re|fctlddn ; 
, to the .Unknown ^to'Siibjiecl, to ^rson ^^ith a''fii^hf nM4 tive Ki 
loading on tHev^(Sjnj4j|fftsp^ have^e reyeAi^Bl^ts. J^V 
For the third person faitor thAe is m mteresting imerentiati^^iwn v S 
bfctween the fntaVper^nit and the {Inanimate situation ifadior^v A per- ■ 
son high on this jhijid i^tson^^ be exhilarated bysti?t iht<gi^\ :: 
personal aft^ vhaVe >Ef Hfg^i^^^ for the Inonip|tev 
person wquld' haw n^^t^^^ the Interpersohel situation^.: c 
and about ayirag^'^ In contrast^ for the Ina^^ate:\: 
situationi, fte-;:w6u^ and have positrve S>u- ^ 
tonomic rea^tibr^, l^i^^^^^ Subject situations are more lite );v 
the Interp^pbal/situatio^s tp j but the reactiphS are . 
not quite, as ex|i^eine/ : ' ' ^ " 
/ITiese rfesulf^^^^^^^a^^ rather commonplace obser- 
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Ii64 Invjiiitidiial Conferenet on Tfsting Froblems 

vatlons as to different of people: one, a very shy person; two, 
a pferson who Is generaUy exhilarated; and three, a kind of person 

of situations. I iubmit that the existencerof these types of people 
rmiei real problems in the^ development of psychological theory as 
related to the situation and the responses of Individuals to these situa- 
tions. The psychological theory must be sufficiently complex to take 
into account the various kinds of individuals involve^. It does not ,; 
appeM- a ^plifle^^er^l theo^^will be adequate, A postscrip|^ 
thSt must be made fof^bis study Is that the data are the ratings byi^ 
the subjects when they imagine themselves ii^^thp particular situations. 
A mucb wr§ e^gtensive study in which the'mdividuals were aG|ually 
plap^ jtt th^e situations and measures of their responses mad^^buld 
be^ifeable* v^,?' 

In the three studies reported, the factor matrices for modes 1 and 2 
tend to confirm previous results and observations. The values in the 
core box, however, tend to bring out newer statements of relation. 
One might take the view that tHe=mod|e 1 and 2 factor matrices are 
dealing with relationrof a moFe^su^e^ type and that^tfi^ core box 
is dealing with deeper and mdi^/i^u|t|4 telationi^ 
the core bqx should have m&re pne^i; ^Kenomena 
■'being observ^d^ One mi^t argue, then, that t)ie ieia|phs^ indicated 
in the core boxes should be of greater general intemrt^ff^Wderstand^ 
ing the phenomi^a. It is my belief that the core boxes for the three 
analyses presehted do indicate general effects of considerabje interest^: 
in the content area pf the observations made . v > 
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^ T#iBhfiology and ^ 




Om^ IDiavyam Moore 
Rutgers^Th§ State University ^ 



Ocoasionally. whi^ having luneh fii the faculty club, some frigfld in " r 
theV^hysicfd sdences wants to^ talk about the prospects of the be- ^ 
havioral sciences. Tlie questipdSf which are raised, often have sharp. ^^-J 
edges. For inst^^; a phyiS|ist onct asked mi wl^ther I thought it X . 

Wwas^Sto^that have caught up with seve^^ : 

^'teenth century physics by tlfe end of the twentieth century. He went 
on tb sympathize With our difficultlesv adding that it must be trying' 

. to sit arpijnd w^ting^and #iting for our **dalileos'* and "Newtons,"_ 
to say nrfthing^about ot^j "^steins;" 

I att^t to parry suchlthrusts (jocular, we hope) in a. variety of 
W^s, One line of defense which works reasonab^ well ^es as foU 
lows: Success, iri behavioral science research presupposes a high level 

. of competence In the phyiical sciences — pur work requires the engi- 

' Waring application of basic discoyeries in the^hysical sciences. These 
disco\^ries and resulting applications have been wery slow in coming, 

^Worel^ecifically/the behavioral sciencps, for (heif proper develop- 
ment^^ must have subtle, sophisticated scientific instruments. Entities 
'as complex as human ^ings cannot very well be studied with the help 
of the unaided lenses ^d intellect w/y,' ^ ' / 

As;a case in point, I sometimes mention digital computers: research 

• instruments which are now playing an imgortont part in the be- v 
Havioral sciences. Such instruments are vital to the processing of com- 

:^ex da^, and are indispensable for the rapid simula^on of ;human 

am indebted to Alan Rmd Anderson and Richard Kobler for helpful criU- 
cismi of earliir drafts of this paper. 
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lunctioni such as higher-ordef problem solving, I elaim that there "has 
' been an uncpnicionable daldy in thrflS^lopment of digital computers 

, .^iiV. aftfif all, the bnnic idea if! sn old nng. Tt w&S-fally wgrkg 
4, tween 1828 and 1839 b^ Charles Babba|e,m professor a^athematici ^ 
at the Umveriity of. Camlwldgeivhe called his machine an "analytic 
engine.'- Hpwevefs ma a mechanical qontrivancg ii was a failure, The 
clunMiJ^ v^heell and cards whic^ he; sought iof iii© were too slow and 
unreliable. JHad the; physical , scieri^ done their ^ork so that there 
was an flectronics industry in the early 1800'Sj we would have had 
more than 100 j years df simulation studies. )|ftd think of what the 
analysis of all those data might have meant to the development of 
emp^cal research in the behavioral sdences! But* alasi we have had 
access to high-speed computtrs only since Wprld War II, and most 
of the time the physical scientists themselves were using them. As cafl 
be readily imagined, it is not difficult to furnish examples other than 
that of the digital computer so as to fill out any conversation and 
. exhaust the patience of the interrogator: there are many other tech- 
^nological innovations, for instance, carbon- 14 dating, now being used 
' in the social and behavioral sciences, which were only recently derived 
j;:^ from fundamental research in the physical sciences. / 
Nevertheless, even if one succeeds in putting off one's colleagues, 
the nagging question renriains, '^Wheri will our 'Galileos' and 'Newtons' 
appear?" What is more disquieting, if serious thought is given to such, 
questions, Galileo's experiments, which were important for Newton, 
did not require complex instruments. As Alfred North Whitehead 
noted, "So far as experimentar skill, arid delicacy of apparatus were 
concerned, this expefinient [Galileo's exf^riment showing that bodies 
of diflferent Weights, if released simultaneously, would reach the earth 
together] could have been made at any time within the preceding 
5,000 years*' (1 ). Galileo's and Newton's triumphs depended on clear 
reasoning and mathematical imagination*, not on sgientific equipment. 
.>|jMhould also^ consider the distressing possibility that these giants 
ml^^have been thrown off stride had they had more precise instru- 
rnents^^ — -with greater precision of measurement, they might have^ 
' discovered that their theoretical model and empirical data did not quite 
match. They were saved this embarrassment by the crudity of their 
scientific instruments. 

It is certainly true that as the physical sciences developed from 
their early striking successes, they came to rely more and more heavily 
upon a complex technology in the conduct of their research, a tech- 
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; TOto^^W Experimfntally d^eloped machines ; j 

hssfm offen btirt a ,Wnuilui to Impprtant theoretical work. As L R. 



phenolwlii invdftihg he^ prassure* vaporization^ and condensation 
Q^jut id^B^ trefnendous i^iprtus to. 

tl^A^ify;^y^ pnerai'sclenct of thermodynanMCi" ^2). As 

' anotlvirakaniffe^'p^^ traces the development bf jnformation thfeory 
. as a r^s^nst of efficiently enqpding mesiages in the 

- ; -ift bifiit 9or^^^ formulations in science have, 

arisen pit of our atternpt to understand and to control man-made* 
^a6vice§--Tj©vices which little understand- 

of their principles of operation. Thi| latter ^int bears further ; 
\ emphisii^ Benjamin FHnMi kite experiment contributed to our ^ 
understandii^ of thi connections betWeen llghtnipg Bnd electricityi 
/ but Swf ^br presuppose a theory which up p fled to boih\ only a l^nch. 
; \ It leemi reasonable to condude frcrt^ history of science, that 
: there is a partial interdependency between theoretical formulations, 
■ ^aitableJ^ciintlflc initrunvenft^^ t^hnology. Sometimes pure, ab- 
-straat^ i^s^ing goes a Jong^a^ even in the absence of elaborate 
'Equipment; JomftoesA^^ inventions thethselves are the sub- 

ject Jnatterbf theoretical Investigations; and sometimes clever inven- 
/tor^^eate us^ful things which :tiS^ do ^to^ understand fully. Nonethe- * 
less; three things stand out as fii^ as the behavioral ^iences are con- 
^ cernedr 1) they:liave been Marked by the absence of mathematically 
rigordus theoretical breakthroughs, 2) they have not produce^ much 
in the way of practical inventions, and 3) it is only in recent times 
that they have benefited substantiailyvfrom developments in mathe- 
; matics, and ffromjenginecrihg application deriving either from other 
sciehtiflc,0isciplines or from iHq contributions of free-lance inventors. 



II 



At this point I should like to turn to a discussion of model construe^ - 
tion irr the behavioral sciences, and some of the relations between 



Yl^ay ^'helped create" rather than **grealcd": thdrc havc' )bc^ thD^sands of 
inavoriQk invent^ who stood puUide .the scientific cslabM^hmeni w^ile Wink- 
ing importaht cohtribljtions to ah. evolving technology— witness Thomas Alva= 
"Edison,. ' ' ' ■ . ''/^'^^j,. ' > ■ ■ ' 
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model, construction 'and technology. I suggest that the kinds of 
questions my colleagues in the physical #ienccs are asking, and which 
I have 'asked myself in the past, are 'misleading. These questions take 
it for -granted that model construction in the behavioral sciences will 
t^ke the same general path that has been takeh^in iho physical sci- 
ences and that these models (though there will be substantive dif- 
ferences) wili first be formulated in the usual axiomatic manner: 
theorems wilrbe deduced from them in the usual way, 

I will argue, on the other hand, that we have been looking in the 
wrong/f^ce for theoretiq^il leads, and that we have not recognized 

'such tWoretical structuriis as exist even though many of them have 
been in%Jain view for%ost of human history, to put the matter an- 
other way\ we are ve^y much Jike the fish in the story about fish 
becoming jcientists. Someone oncu cdnjccturcd that if fish were to 
ever'devclfe a scientific culture, the last thing that the scientists 

'among them would discover: would be water. In a similar way, it is'my 
position thJt our theoretical ideas about human bthavipr arc so largely 
ingrediep^ of the medium in which wc live, ourselves included — we 
are, ^^u to speak, made out of models ot' ourselves — that we have 

-gr^ difficulty in distinguishing beiwce^ ourselves and our theoreiica! 
formulations. We not only swim in a medium of abstraction, but un= 
like fish, we are in part made out of the same stuff.= 

Let me turn to the task of specifying the major outlines of the point 
of view about' model construction which I have in mind. In a number 
of papers (7) (8) (9), Alan Ross Anderson and I have urged that we 
cease looking at the ordinary human being as, an' ^nheofctieal or a 
/;o/;theorctical or even an'a^r/thcoretical cruaturu. Behavioral scientists 
generally take the stance that the ordinary- nfah, a citizen in good 
standing in whatever community he may live, has very little in the 
way of intellectual resources to guide him in managing his affairs. 
He is credited with ha^ng soriie folk sayings and aphorisms. Behav- 
ioral scientists are quick to pint out that even these blunt conceptual 

- Admittedly; the thesis which I am advancing is coimtcrinuiitiva. It also may be 
dead wVong. I invite you to consider it bC'L.iiisc it is a pcrNpuctivu from vvhich my 
colleagues and I have been able to make a number of inventions which are use- 
ful in the behavioral sciences, for example, the talking typewriter (1), and it has 
been productive of some formal work in mnthematical loyie (4) (3) (6). No 
claim is made that this perspective yields, these produets as strict deductions 
from an axiomatic scheme nor is the claim made that it is the only per- 
spective which woijld have yielded these same results. 

6!) 
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tools are generally contradictory. For mstance, ^^Absencc makSI the 
heart, grew fonder^* and "Out'of si^t, out of mind." In addition to 
theae saws, "the average man^' has some practical knowledge, some, 
skili^at ruIfe-qMhumb reasphin and if he is really hard pressed toout 
something,*he can fall bdck on various tradition-base4 explanations. 

We would like to suggest a 4pftnf view of^'TVlan;' We bplieve 
that early in l^man history, probably at about*'the same time rafln 
^developed mtural languaps, he also Areatetf^odels of the.ino|t im- ^ 
"pprt^t features of his environment. These were abstract models, 
which collective covered relations holding be 1 ) ,man and 

nature — iniofara* nature is nbt random =^rman and the raidom, 
or charicy elements in.exgeriencc, 3)' man in his interACtiori#'rcla" 
tions with others lijke himself, and finally, 4) man and the normatiy^ 
aspects of gr^ living, Structufe falling within these four giasses'^f 
mgdels were created by unsung Newtons, so that there dpe^not pxist 
a society, however primitive, that does not have cultural o%cts fall- 
ing in these four lategories of mndels. [For an explication of the-con^ 
cept^culturalobject'^see (10).] ' * ' 

f Every society, as far back as we have any evidera:e. hm^ puzzies ^ 
^/hkji stanrf in an abstfact way for man^natur^lations. Every s^iety ■ 
ha? some gllmfes (?f chance. (According to 'view of the matter, 
games of chance arc absfftict models of the aleatory Aspects of^exist-^ 
ence.) Every society has g^me^^ of strai^s^ m'hhe sense of von 
Neumann ill). These games capture aome of*the j^cuHar ^fures 
of intera^onal relations among meiK, relations in which no pty to ^ 
an encounter controls all of the relft^ant variables upon which the: 
final outcome depends, though each contrd^ some of thestf variabl^^s 
and each participant^can take account of the potential act^s of 
Dthefs involved in the same situation. Ever); suci-ty hii^ ae^ihetic 
entities- art forms, which we claim givo. pdbplw opportunity to 
learn^ to make normative judgments about their experiences. All'so^ 
cifeties make use of these cultural objects in the socializatiort^f the 
young and for the re-creation or recreational enjoyment of those who 
are older. Simple forms of these models urc internalized in childhood, 
and more complex vtfj^ions cif them sustain us in adulthood;^ 
_• - * ■ I 

3 It should be pointed out that until mathenuiiicianH created formal analyses 
of the structure of some of these modyis. therr dypih unfd subtlety were not' 
'- 'mpreciated fully. Of the four classes of modcU, two have rceeivod adequate 
formal analyses, namely, theories of probability hiivc all games of chance as 
models and the various theories of games of strategy have all games of 
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^ • From the pomt view presented herej the ordinary man in any 

= society should to thoy^t'of as having in his mind (if you are of a 



^ school of Aought w^p grants him one) at least four classes oj 
ilbdeis^htqh he can use in, a highly abstr-act way to guide his be- 



havior.^rom this stai^oint, it probaWy would not have beerr pos- 
; iible mankind to develop corn|lex civilizations U^ithout thqse atev 
^ . stract fol^^odels. We nfee tht assumption that having these models 
\^ was a necessiry condition £oi*the development of complex s^ietie% — 
M^itbout these a^l^ttions, man would have been imprisoned in small 
g^upi; not unlike bands of especially intelligent baboons, * 
;^ But hisiori.^% Speaking, man not only de^^eloped these fascinating 
. conceptual structure; he also devised suitable techniques for seeing 
I" to it ih%. they were metered. 4 would like to pomt out that, for ttfe 
; ^mos^.part, one leams, but is not necessariiyAtaught, to play with folk 
modlfs. What is taught are ihe ''fules of the game." Once the rules 
x^:. Ire understood, fcach participant is largely m his own except 'when 
vt^ the models are perverjed by professionalism. :ii ^ 
\^ In ev^ry society there are sodal norms which distinguisli between 

'0 serious mattep, on^tht) one harfd, and fun and gardes on the ojher, 
f- "^^k Generally, specific times and places are se!' aside for the enjoyment 
mf of thes^fdlk'' models. ^Iso, the state for winning or losing are keprt 



strategy as m^^s. The formal treatment of f^uzzles is not in as satisfactory 



a condftion as^fe games of chanee and gan^s of strate§y; however, we have 
" ^ suggested that the^^^^rie^^f natural dediiction may'be appropriafe here (12). 
' ;When it comes to 'Esthetic objects^^vc:> one is at sea and it is not know^ 
* 0^ whether fot^mal anal^e*-,.of aestheff^ objed'ts, should such analyses prove 
possibltev -Will result in of^| or more distinct classes of theories. We should = 
i?car in/nind that until tfife work of von Nfumann, no one was ^in a position 
to drttvt a .^heor^tally cogent distinction between games of chance and 



games' of stf^fegy. 

^ It^ertpinly is true tl^ the ordinary citi|en% the last person you should ask 
%bout how ^he.^sizes up situations and r^akes decisions. He lias had little 
training in ^eoretical discoi^, and ^ required the highest order of genius" 
to cart^ through smmd for^Sl analyses of these familjar^ulturar products. 
It is inter^ting to'r^fe too that none of tly.^four classes ^ 'models calls for 
very, much in the way of tfechnologfcal expertise in fashionmg the equipment 
used in con&tion with themj—for example, bits of wood will dO'for checker 
ecei. It is clear that the ptlj^sieal Side ttf these f^ moSels is ti-ivial, well 
within the technolftical competence of tarty m'an. » 
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at some noirtln&l value insofar as profit and loss eQter in. In addition, 
there are norms which regulate expressions of feeling and emotion 
with reference to using these folk models. During the course of play- 
ing with a model, one is permitted to experience a fairly wide range 
of feelings and emotions, but extremes are excluded. These models, 
as it were, serve as a school for emotional expressions — this, is the 
kind of school in which boredom is unlikely and uncontrolled emo- 
tional fren^ is forbidden; All in all, the set of norms governing the 
use of the models knd Ihe models themselves have proved so suc- 
cessful that people havg to be prohibited from playing too much, 
despite the conceptual depth of the materials with which they ^al. 

Now, someone might be tempted to ask the following question: "If 
you think so highly of these models as guides to action, and if they 
indeed represenf abstrsctl^ so many salient features of human exist- 
ence; why should we not rest cojitent with them?'' My answer is that 
these models did indeed serve man well during^ most of his history. 
However, there is something radically wrong with them with respect 
to their present theoretical relevance. So long as the ordinary man 
livedLput his life within the context of a static social frameworfe these 
' modek matched his world —the models themselves are essentially 
static ^titles. For instance, in any play of the game of= chess the rules 
— that\is, the boundary conditions — remain constant, There may 
be plenly of lively action going on within this stable frame of refer- 
ence and the participants my feeL| wide range of emotions, but the 
rulfc are both fixed and inviolable, in a normative sense. If you are 
woRing a puzzle, say a jig-saw pu/zle, the picture fl© be completed^^; 
does not change as you work on the puzzle, and the pieces pfeserve"^ 
size and shape consfahcy. If you go to see a play two nights in a row, 
with trivial variations, it remains the same play — the' actors do not 
change their lines because you have seen it befoccv though you may 
appreciate it more thoroughly on seeing it the second time (assuming 
it us an interesting pjay). 

* ii 
V 

^ The basic point I am attempting to make is ;hat the folk models 
mirror the static qiiality of unchanging or imperceptibly changing 
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societies. The folk models in this respect are like the ^^fewton^n v6n= 
ceptions of space and time — both presuppose a frame, of fsftrence 
which Is invariant with respect to all that goes on within them, 

I scarcely need to remind you that we no logger live irt such an 
absolutistic universe. Since World ^r II there has been a massive 
acceleration in the rate of social change — - an acceleration so great 
that it is difficult to graph it. This fact is now appreciated by most of 
^ us, although we are^ not prepared as yet to d^al with its consequences, 
Frptn this standpdi'nt, it is not inappropriate to divide human history 
into two major periods, before and afte^this rapid acceleration got 
under way. The primitive period of human history ends in the 1940's 
ajid the modern begins then. It really dqes not make much difference 
what scientific, technological or social function you care to consider 
~ each has gone off the graph and is shopting upward. We now live 
in a truly dynamic era and the most serious question which faces us 
is whether we^can maintain a stable dynamic equilibrium. 

Turning again to folk modelsy it is my point that they tend to 
inculcate an abstract conception of the world which is incompatible 
with a civilization in acceleration. The chaUcnge is to create new 
models appropriate for these changed and changing fclrcumstances — 
we need dynamic mo^Jcls — and these models must pass the striilgent 
test of being enjoyable to their users. It is essential that people have 
to be admonished to put them aside, 

I have given considerable thought to the problem of devising suit= 
able scientific models to replace the traditional folk models. [See (13) 
as a first effort in this direction.! The "rules of the game" must be 
changing ones, yet it will not do if the rules simply change arbitrarily 
or irrelevantly. The successive sets of rules must bear some plausible 
relation to one another and to what goes on in the course of play. If 
you give sejious thought to the problem of devising dynamic models, 
and?! hope that you will, 1 believe that you will come to the conclu= 
sion that it will be very diflicult to create such models without exten- 
sive employment of the latest developments in technology. We cannot 
interrupt the players over and over again to explain new sets of rules 
— it is no solution to substitute one fixed frame of reference for many 
fixed frames uf reference. Instead, the player wfll have to discover that 
the old rules are, for some reason or another, not quite working — he 
will have to find out for himself that something has gone wrong and 
induce a new modus operandi that will suffice to meet the new condi- 
tions. To choose a trivial example, the jig=saw puzzle again, I do not 
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see any way of making a dynomic jig-saw puzzle without employing 
some fairly complicated electionic equipment. 

The final point which I wish to make has to do with testrng. We 
not only want to create new and more appropriate models, but it is 
important to make certain that they have the desired consequence 
of helping to fashion a highly abstract, open and creative approach 
to the solving of dynamic problems. If we look at the development ot 
tests from the standpoint that has been explained here, we can see 
that generally speaking, they fall in the category of the puzzle model. 
Various I. Q. tests are cases in point. These fcsy consist almost en- 
Urel^ of sequences of short staUc puzzles. For this rcasen, assuming 
the folk-model analysis is correct, we should not be too surprised 
if the assessment of peoples' puzzle-solving abilities should have some 
predictive value for their behavior in everyd.^ life. However, since 
most tests in current use do not measure skilf in handling the other 
three classes of models, we also should expect a good deal of unex- 
plained variance — and we are not disappointed. . r 

Even if those who are interested in test construction were to build 
tests for the other traditional folk models = a demanding task m its 
own right — the opportunity srill would exist for building tests fw the 
projected dynamic models. Since these models are, for the mostgirt, 
simply speculative notions at the present time, there is 'an offrar- 
tunity for model construction and test construction to work hand in 
hand And since the new models would have no traditional warrant, 
it is doubly important that they be tested as they are developed. It is 
my opinion that such new tests, like the models themselves; will re- 
quire a full employment of our technology. 

To recapitulate, let me say that there Is a sense in which out con- 
ceptual grasp of what is now the 'subject matter of the behavioral 
sciences exceeded our grasp of what is now the subject matter of the 
physical fciences for most of human history. The folk models created 
by now-'forgdtten geniuses represent thcorcticiil schemes which match 
the modern constructiofts of the physical sciences In subtlety, rele- 
vance, and applicability. These folk models. are, strictly speaking, pre- 
scientific creations whi|h were essential for man so long as he lived 
within the framework of fundamentally static social orders. They are 
antithetical, however, to the needs ofyicieties in rapid acceleration. 
It 1% suggested here that what is wantcd^irc models that are dynamic 
and will help in the creation of n(j,stt.,£ipgTpaclies„t^ problem solving. It 
-is diffieult to sec how to make 'such modALft without the help of ad- 
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vanced technology. These mpdels also most, be tested and this require^ 
ment has many , implications for test coristructlon. 

As a final thought, it may be, well to reoall the words of the per- 
ceptive Henry Adams. In- 1905 he stated, 4n the context of a dificus- 
lion of social acceleratlorr, that this acceleration 

prolonged orte generatioij longer —Vou Id require a new soclrf. 
mind. As though thought were conimon salt in indefinite solutidn 
it must entei^anew phase subject to new laws,, Thus far, since 'five 
or ten thousand ye^r^, the mind has successfully reacted, and noth- 
ing yet proved that it would fyi t6 react but it would need to 
jump (14). . " , - " - *^ 
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There are several vantage points for viewing application* of measure- 
ment to problems concerning the culturally disadvantaged. An obvious 
one would be from within the- science and technology of measurement 
itself, which is the context of this conference, the bias and. the avenue 
of contribution of its sponsors, and the vested interests and profes- 
sional legacy of its participants. Here, one might rush pell-mell but 
bravely into such matters as culture-free tests or the growing inventory 
of specific abilities and the search for their functional utility. 

Another vantage point wourd be from the frame of reference of 
the culturally disadvantaged, ps a significant social force. This would 
involve, of course, our broader societal concerns with such problems 
as urban poverty, minority youth, civil rights, integration, unemploy- 
ment of marginal labor groups, or manpower training. The urgent 
social and economic forces and the concomitant national concern lie 
most certainly at the core of the recent surge of interest among social 
^?:if^ntists; the same forces dictate solutions much more frequently 
than the efforts of social scientists. A sociaUhistorical perspective may 
therefore provide a rapidly shifting dcrmitiqn of "disadvantage" which 
is sometimes a minority group problem or sometimes a function of 
economic and population trends that are shaking our social order. 

No discussion of cultural disadvantage among psychologists and 
educators could ignore either the social-historical domain or the meas- 
urement technology which we eniploy to identify and describe dis- 
advantaged individuals. Yet there is even another vantage point which 

* The writer is indebted to J. A. Davin for critical review and editorial sug- 
gestions. 
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would seem particularly usefuU for it brings to light certain crucial 
Issues which have pervaded the rapidly burgeoning literature on cul- 
,turardeprivation and at the same time reveals how effective resolution 
of thosd issues has been conspicuously by-passed. This vantage- point 
is that of the counselor. 

The term ''counselor" is used hereun the generic sense of the pro- 
fesiional in schools and in public or private agencies who is con- 
cerned 'with the educational or vocational development of individuals, 
As such, the terra subsumes many change agents who are dealing with' 
the problems of human deprivation and learning deficit as weir as 
with the compensatory programs designed to reverse the process of , 
deprivationi; But, the unique aspect of the counselor's role, for^our 
purposes, is that he must.be dedicated to optimal development for all. 
Th& all is now begrnning to include, in practice, the culturally dis-^ 
advantaged. And the counselor must now view the. disadvantaged a's 
• individuals who are underdeveloped iii one way or anc^!||r^ 4 

It is^not the purpose" of the present argument to suggest tliat there- 
may be something ntagic in the counseling process which yields a 
brave and competent new self where fear and impoverishment cxisted^ 
^previously. It would also be patently absurd to attempt to show that 
counselors have* learned more about their measurement tools than 
the psychometrists who have developed and researched them. Nor 
can it be maintained that the typical' counselor has a ruch background ^ 
of experience with the deprived, or that he is free from negative 
biases which Result In a devaluing of members of groups who seem 
unable. to achieve or to produce Jn standard ways, Counselors, like 
other helping professionals, have been employed by, and addicted to 
the views of, the ''pwer structure/' In fact they now find their job 
more difficult precisely Jbecausc it more frequently includes the dis- 
advantaged. ^ ' 

Today, as a nation in the midst ofi an ideological crisis, we arc 
revising our premises about human potential as a result of social, eco- 
nomic, and moral resolution rather than new discoveries of unusual 
abilities or of new 6utl0ts for traits heretofore Jacking in application. 
White the general public has other options, it is the counselor's func- 
tion to guide his counselee tlVrough rebirth and regeneration, if neces- 
sary, and to help him find a place in the social system where his de- 
veloped capabiliyes may be utilized. Face-to4acc with disadvantage 
in the most personal sehsc, the counselor fulfilling his function must 
ask what Wasurement offers beyond the somewhat painful confirma- 
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tion that disadvantage exists in measurable units and that the going 
may be rough. ; 

Measurement is essentially a matter of science and technology. It 
is good science to describe and to attempt prediction from samples of " 
behavior^ permitting thfe objective findings and probabilities to guide 
further action. We may measure air pollution* for example, or fre- 
quency of smoking and predict the incidence of cancer, or we may 
observe the effects of harmful food additives or pesticides. In the 
application of measurement to the disadvantaged, howevet, we go 
beyond scientific issues. Although disadvantaging physical^'environ- 
ments .ajfe clearly revealed through scientific research^ the social 
impact of^ the proof provided by science can best be summarized as 
provocative olsome moral "n^dignation and ver^ little St any, action. 
Moreover, the science and tecnnology of measurement of human traits 
are not yet at the level to enabHe us to make individual predictive de- 
cisions for mQSt people except with wide degrees of. latitude and error. 
Thus, with leis than reasonable priainty of prediction, and with ^little 
comfort to be gSined from labelling alone, the issues are evidently as 
much moral as scientific. In the current atmosphere of developing 
the great society, of combating pQverty, of fighting inequity, there arc 
salient issues of a political, socinl, and economic nature. It seems 
to ma that these issues are scientific-technological , philosophical- 
ideological-moral, and po!iticaUecanomic=social, .Let us consider each 
of these, / ' 

The aolentlflo^TeoHnologlaal Issues ^ 

It should be assumed at the outset that theory is powerful and that, 
the oft-proclaimed 'iack oMheorctical underpinnings" notwithstand- 
ing, certain theoretical formulations about humun ability have influ- 
enced practice, 1 he consider;ibie body of data about human abilities 
and differences reviewed by Anastasi, Tyler, and Super and Crites- 
provides good evidence of thr existence and impact oC^theory. the 
more recent works of Hunt and Bloom reveal that implicit or explicit 
theories of human development, espceially the development of infiplli^ 
gence, have influenced measurement, research, technology, and con- 
sequently counseling prLielice/ In rehuion to problems of cultural dis- 
advaatagc, the vast literature on the intelligence of Negroes demon- 
strates hqw seemingly neutral ''scientific'' data and "objective" theory 
are imbricated in everyday practice, ^ 
Even the most basic theoretical formulations have sometimes, been 
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Tesponsible for widespread fallacies in the profession. One example i$ 

with the normal curve, so tremendously useful in describing majority 
populations. We expect, seek, and find those individual^ to fill the 
tails of the distribution^ Yet Tyler points out that ''It is possible 
to change a skewed dis^fibution into a normal one simpWfpy making 
the test on which It is based a little harder or a little e£^^, depend- 
ing upon the direction of the skewness/' This is defensible' tdr mathe= 
matlcal and practical reasons, but Tyler continues: "We know now 
that test scores can be manipulated to give, us any sprt of distribution 
we want." * 
The dipRculty with. the normal curvafrom^the counselor's viewpoint 
is not with the test construction and the norming, however, but Mth 
the inisconceptions that follow, or are fortifled by, the presumably, 
immutable distribution of traits and talents. Thus> intelligence is seen 
as (I) a dimension on (2) a mile. Both notions have been attacked" 
as scicntificallyl^fallacious and practically deplorable, especially by 
Hunt who argues that drawing sarnies of bdha^vior for evidence of ' 
such organizational structures as schemata, operations, and concepts 

' is more scientific than dimcnsionalizing and scaling within a vast 
domain of human functioning. If persons are open .systems in which 

Vchange otfcurs.as a function of unspecifiaWe future condirtons, then 
the use of dimension and scale in counseling Is particularly serious. 

Despite the evidenqe of Tuddenham and others that^ World War II 
soldiers performed considerably better than World War I soldiers on 
intelligence tests, ^ the normal curve as an eternal, immutable verity 
continues to guide pur perception of the ^particular distribution of 
traits in fixed and inevitable proportions. The counselor of the dis- 
advantaged, however, "can only wonder if the ^'normal distribution'' 
.has become a modern' version of the Procrustean bed on which indi= 
viduals have their standard deviations stretched or lopped off as de- 
sired. Thus, for the counsclof of the culturally disadvantaged, the nor- 
mal distribution, despite its utility in other areas, can only reinforce 
the notibn that intelligence is like an oil reserve; there is just so niufih 
of it, and our job Is to dig wells and retrieve oil, using our science tg - 
make 'the operation rhaximal.ly profitable. If I may inject another 
metaphor of great relevance to counseling, we believe that we can't , 
make a silk purse oiiLof a sow's ear, but we ignore the fact that with 
modern cheniistry ^^cai^! We cannot allow, su^h notions to distract 

. us frpm renewed and'modern efforts to dqvclop and transform people 
through education and other institutions, 
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Another problem results from coqf using test' performance with 
genotype or innate capacity. Hus^^ipts: this up most cl^early, His pro- . ^ 
= / posal is that we think ra^er in terr^^ phenotype, that is, already 
actualized capacity. The research studies that deal with the proportions 
of intellectual devplopment' that ^rq attributable to genes ^ are most 
-fascinating?^t they may serve to inhibit efforts with deprived youth 
if th^henotype is confused at my point with the genotype. I fell that 
this iV precisely what happens.^ This leads us to consider the most 
* psychometrically respectful use of tests in counseling, that is, pur- 
poses of prediction. • " ■ ? 
To follow Hunt once Again : 

^One implication of (the) idea that devclopmpnt . is an open- 
ended process puts^ a logical lirailation on the predictive validity 
% of tests of intelligence^ or oh measures of any personal charae- 

J teristic. Prediction in science, however, is always a' matter of stating 

' what will happen tp given objects in a closed system for which the 
relevant characteristics can be and are fully specified:*, . , It 'may 
become ppssible to formulate laws which predict the characteristics 
that organisms with specified genetic constitutions will develop 
under specifled programs of encounters with the environrrietit^ It 
would appear to be outside the realm of scientific possibility,, how-i 
evers to- predict with precision the future characteristics or phenio- ^ 
typic fate of any organism from knowing merely its present char- ■ 
acteristics, without being ^ible to specify the future conditions under . , 
which it will live. .\ _ \ 

It seems to me that herein is to Bt foundj^he central issue of all 
counseling, but especially counseling of the disadvantaged. The essence - 
of the counseling ir-ansaction is the opening of the system — the phe- 
nomenal system — as well as the opportunity structure comprised of 
educational arid vocational resourcei. This view of the counselor, t\v . ^ 
only view tei\able under our working his role, places gr-eat 
importance on testing for diagnosii itUt^^^sLO prcdic^ipn. Measure- 
ment should be used *'to improv^.j^fo;ji^'?;'a| Ebcl sugg^^^ 
than "to determine status." '- ^''^^^i^r^^P^^ ^ 

The emphasis on prediction, so wcli-foUndcd for the purpose of test 
construction^ validation, and especially selection; Jeads" the counselor . ' 
* to an illusion about the nature of the "permanent" 'cndowrnfent that 
individuals bring to counseling, This chimera blocks the entrance io 
that mysterious realm which may h6l4^the secrets of the developmental 
and transformational activity which successful work with the disad- 
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/ vanl^ed entaili. Leit we conclude that we lire discarding too quickly 
a rigor wc havt treasure^ in the past, we can also note the rather |iard- 
headed conclusions olTTyler and Ghiselli ]^at aptitude tests have very 
limited pre^ctl\^ value/ th§ success of traditional tests of schb- 
4tstic a^itude for gre^^ting achlev^ent in the popular educational ' 



portedaby Goltoan*s recent review of vocational assessment. The 
single ,tt^ on the one hand and the^elaboratf test battery on the other 
certainly merit ^ jame criticism if eithe^ is geared to bf ief ^-coun- 
selingV Interv^wi which do hot actively ^.engage the client as a par- 
ticipant in the process. Too frequently in the past the fusion 
' s^iM and festingf-ha^ amounted t^ p pattern of four or five hours of 
testi^ and an hour of "counseling/^ With the seriously disadvantaged, 
^ diagnosis and treatmen^t are infinitely more impoVtant than the mis- 
placed emphasis on testing and prediction. And, In this view, the more 
modem emphasis among leaders in test theory and construction on 
construct v^ity (e.g.; Messick's work at its) seems more prom- 
ising. We may fear, however; that it will be some^time before the use 
,;of tests to predict and, in the case of the disadvantaged,jEhe use of 
'prediction to select out, is displaced by the usie of t^ts fordmgnosls 
toward understanding and treatment, ' " 

No consideration of the scientific and tcchi^elogicar issues would 
- seem complete withouj, some examination of the CHlt%e-fair , or 
^Iture-free tests. In the framework of the foregoing disqiission, iBs 
^ approach promises to Be a will o' the wisp insofar as it pursues the 
unproductive searchvTor genotype. The culture 
Is dedicated to the proposition of genotype. If 'we think in^ terms of 
diagnosis, in terms of point-to-point phenotyDical sampling and then 
-. ' of appropriate intervention, the dlefulne^ss^or such instruments is 
minimaL To s^ain toward ^heir deve^&yi^nt is not only to ignore 
/ largely that the majority culture will p^e^H in the environments in 
"^S^ which the disadvantaged must operate, but also seems most clearly 'a 
\ \ pla^tive atteriipt to demand^ proof of an adequate genotype:^ before 
%i^esting in any developmental program. % ■ 

Thm ph|loMphlMi« ld«olQgidal, pnd Moral issues fn MsMursment 

Missing in many of the current policiel and programs that aN con- 
cerned with the disadvantaged is a frjesh view of human potential. 
The ideology of |r^^r ejcpectations must rtplace the seemingly ob- 
jective but quite clearly ideologicai commitment tO' KurdJes and cut- 
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off described aboVe^ 'Hje economi^^ s^^ty mduc^OTM pl^chblo^ 



of scarcity. The econemici of aburi^apffl has uhfortunately given uf ' 



veiy little more tHln^ the psychology of afluence; the- psychplogy of 
abundant ability — of cq|peni |or creating .45velopmerfE^pport^nitfes 
; ■* for all levels along the spectra of abilities a&urrerfU}^ asses^^^ is 
^^il^ Jj^t yetvwU^ it is e^ to attribute the ^alth we hs^ya to 

■^^ ■W&ntje t&^mkot both natural and^human resources* and in'^scriBing 
, iJtoi^l^stSMatlon of the former to the gifted portion tfihe lai^i 
V ^tit we all kno\v, tOOs that selection Is a short-run conce^,^e achieve 
V r tenofpbfary success, as did Binetj m screening out of the regular lystem^j 
\ those who would deter the pra|ram's impact on the '■appropriately -'' 
talented*" Such efficiency becomes a culprit when it becomes co|tly, 
; when it results In the depositing of large groups who can only be a 
burden to the by=passing society. 

The essence of the philosophical,, ideological, and mbral issues 
in the application of measurement to the disadvantaged lies in the 
* supposition that bur basic concepts of human ability have been drawn 
from instrurnentatlon which has been developed tp conform to a 
- IJIll^ particular view of human "nature. This view considers the essential 
development capabilities to ^ fixed, or genetically determinedf'withi^ 
" . only, relatively ll|ile change through environmentarinfluences possible 
after the secpndrfryilrd year of life. 
^ ^ Thus* the. counselor, engages in a self-fulfilHng prophecy: Starting 
" wlth ceftain assuthptio^^^^ a,bout hurtian nature, about racial groups, 
about the etiology op disadvantage, he may use the. genqtype as the 
ynodel of individual development. The great temptation in ,counseling 



Is to inventory the sort of pragmatUE solutions that society has pro- 
altering a process wftich has always seemed irreversible 



vided, rather than tp enter into^^t^**teal effort that is involved in 



Another related Issue is that we dre so committed to traditional con- 
cepts and fc^piulations and are so persuaded that change is slow that 
we fail to not^hose events or scicniific researches which show radi- 
cally difffcrent concepts and iormulatlons ,to be vital, dynamic, and 
useftrl. FOf example, maTiy people in this country harbored the illusion 
that' the Soviets were mechanicallf^nepl; we know,. because it is ii part 
of our own recent past, what happened to this attitude when they put ^ 
Sputnik up befg,re us! The evidence provided by Margaret Mfcad % 
of the rapidity with which cultural change In the S^uth Seas mpyed ?l 
people from the Stone Age to the Jeep are even more germane, f^ore, 
to our pomt today are the. Iowa studies, much marigncd for year^, on 
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the peat accelarationi that CQuld be induced by conscious, planned ^, ■ 
jproeesses in cHild development; or, to the smie goint. Kirk's studies* 
with mentally retarded children. In a yePunpubhshed study which I 
have been conductihg* we have found ra^ and sipiiflcant increase in 
IQ fbr J 8-year-old ekperimentals (disadvantaged youth in a work-study 
prd|rMfl) over sipto cont^ls in a seveii month period. Therevii 
indeed mounting Widance that Jntel^^^^^ is far from immutable. 

Beyond these considerations; ti^ete^s still another difficulty. Wfi^t 
isn't answered by increases in ^ the fact that increased intelligence 
4^thout ready opportOnity to function Is not much oj an outcome. The 
^ft^ij^ically perfect plant can be^encased in aft otherwise perfect 
environinent, but without a window'to admit the sun there Cfn be no 
tropi$mj\rhe^unse!ee; however fortifted with training or increasfirf 
capability to deal with the environment, still needs a window before; - 
self-actualizing TTiovement can take place. The heliotropic client is the 
kind of succeW story everyone is looking for. We revere our notion 
of the individual who^ despite every handicap, desprtte every conceiv- , 
able kind of buflfeting by fate, emerges at the end as invincible. -The . 
windowless room would be ^or botany. This gross distottioii of 
Darwini|m is poor psychology. Yet this is the dominanifj ideology of 
limited Apectatloris. • , 

*We know the jokes about the clerks who were made cooks and the^ 
cooks who were mad& clerks through the aptitude tests administered 
tp kiducteii irrWorld War U, But in real life, these people could, and 
did, funqtlon in the new foles. Super*s basic premise over a decade 
ago, that ''although each occupation requires a characteristic pattern 
of abilities, iriterests, and personality traits, there are 'tolerances wide 
enough to allow some variety of individuals in eacl^ccupation and ^ ' 
some diversity of occupations for each individual," rriay be an under- 
statement. We may recall Jones' classic work which exposed Jhe fact 
* that the presumed decline in intelligence of the aging was an artifact 
of the ^ats employed. Our commitment must be to the individuals 
and thS job; we need to I90H for capabilities to occupy produttive roles 
without the Testrictiven^S jwhic^ pur conventional notions ofaesting. 
and fitting may Impose. While such examples may seem remote, we - 
have the evidence ffd^ RosenthaVs studies of "experimjenter bias that 
even the most careful researcher tends to get the kind of results that 
he wants. The irifltfence of.ldeolo|y makes the counselor susceptible 
to the same kiW of biases, only in his case he perpetuates a wide class 
of human behavior. 
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The pFobleiris ot disadvantage are frequently matters of power and 
po^if itruoture, with the disadvantaged being powerless 9$ well as 
deprived and poor. Nominal or real disenfrarichisement is often ^orre^ 
lated wAth^ ffllnQrlty group status- and pobr education with non^al or 
real segregation. These economic, political^ and social issues are in- 
vol^fd in meaiuremenl when, through pupil placemenf laws/ tests are 
usw to fortify ^ social system. The de /ecra^ segregation of students 
by school and.by ability groupings provide significantly lower levels of 
expectationi experience, and expression for teathers and students. The 



W^^hon tot the students and serves to close rathef %tHan open the system. 
P??^;' The |actis that we can now afford to train ^d to develop people, 
YW?and we cannot afford not to train and davelop people. That political 
Ij^-^ leader and friend of education in the Souths O. Max Gardiner, Js 
^ reputed to have said that **Good education is expensive — in fact, the 
only thing jpife expensive is ignorance.^* Duf ecqnomics of abun- 
dance ttiu^rtSeT&^tched by a psycholo|y of abundance. We arc obtaih- 
ing^e financial me^ns for the programs and'fo( the research simuU 
taneously; the research must not lag. ' - u_ « 



ImpllGBltons for Ihs N^fsftslortsf Role of tOOunselW 

The three areas, of concern* about measurement issues in work with 
|he disadvantaged fuBe nicely into an indic^on of what the scope of 
the proTe^sional role of tde counselor must be.' As a 'pragm^c scien- 
tist, he cannot afford to divorce his focusing on!a paE4cul^pro^em 
fronUhe^alcfititey. The striking thing abp the really great physi- 
eistrf^hp have made im^rtant contributions tg modern ^epry is their 
^"generaKTcnowl of total ^nature and their" pervasive fnoraLconccrn. 
Thif^^^iU^be th^ructol factor in determining how lon^'it may be for 
jhe morality to Decdme actuality, , 

Chalincey and 'Dobbin's discus,sibn o^ guiddty^ American 
public schools is a most eloquent statement of whtit; wft/K be a jood 
model The way in which testing is harnessed to guidance, however, 
confines the utility of thip model to the rnajoritiesMr^cs not fit the 
conception or the practice of counseyng with the cultuplly disad- ^ 
vantaged, Indped, the role in which measurcmcni in pdunseling has 
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been cas^y. Chauncey and Dobbin Is one of. ameHoratioh. Perhaps 
coming to grips^with reality (and using ^ests- as ohe^ourcevof mjor- 
mation about important reaUties) is the. first step m chan|ing^those 
realities that the individual does not want to accept. With the disad- 
vahtaged. hSwever, we are concerned about the Kc^lities of poverty, 
ot segrega ti on, of m m Lomeriangsr^^feouFi^^ 
ing nbrmaliy provides counselors wears thin slmplyil^sc the prob- 
lem is so vast. ■ ' ' ' . ■ ' 

The great* bidto|ist Dubos has 'pointed oot that laboratory science 
in his area is ihsufficient unto itself to ease the ills tow|rd which it is 
directed. He says '"The most effective techniques to avoid, disease 
came out of ^e attempte to correct by social measures'ajid by sam, 
tation and public health. The introdoclion of cotton undergarments , 
easy to launder and of transparent glass . that brought ligl^ into the 
most humble dwelling contributed moro-'to the control of infection 
than did all drugs 'and medical practices," Modern ^robiology is^ 
superb, but it is poor sanitary practices tbat^^re respoViffible ioj pm- 
oic outbreaks of -staph. Even If the Cbftdncey^Dobbin, model and 
the Merttal Measurements Yearbook give us the illusion that .we have 
a pharmacopeia, and though testing as ritual seems to cleanse the 
counselor and' the client at once, infection itilP lurks just outSldc^ 

the door, - / , P, U* 

In this sense, testing ^ould appe.-^ to be very much |econdarf to* 
working with the individual and through the environraf nt to improve 
the opportunity structure for^the disadvantage. Om #es not have t^ 
ignore totally *he knSwledge of hum^ difTerential piychcHogy that we 
havl acquired. The assessrttnt of, say, a nineteen-year-old high school 
dropout as having an IQ of TP, a 1 1 « gradeflevcl ^ore on paragraph 
meanfeg on *he StAford Achievement Te*, and^a 7.8 ^ade^cvel 
word meaning score provides, some very coeent suggestions about the 
way this individual functions^and may aidfn the design of remedia= 
tion^ctivity, or bitter, of developnfintal aitivity; But still, another 
factbr is thM few practitiMers of counseling have the knowledge and 
exAience requirj^ to use tests # this manner, The use of the most 
gross measures, the reliance, in administrative decisions, on the global 
score (and this is what ictuallj happens in the long run), is contrary 
to the cotnselA's purpose of increasing human variability. 
« I am not impressed By the evidence, therefore, that statisncal and 
actuarial methods are superior to clinical or instructional tech= 
niques in counseling«activities with the disadvantaged. Here the issue 
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is^eitonabus wd simply not appropriate. Tftfting in this view is pre- 
Uttjlria^ and incjd^ntal to helping, to modi^ing, to teaching, and to 
reH^ucatm|; tl^ view of testing as^act-finding and prediction; I mairv- 
taiflVlaads to Ae butrtisSing of initruction. ^ ? " 

Tile assessmwt-apprars^^ necessa^ bU t^tas uflB-- 

citrit to performance of the counseli fl l function in wg rfclMLlthe 





disadvantaged: TTie auto-educktiop model is necessa^ ana 
The defiiion idiodel is spjurious in this instance becalse b:.^„.- j^. 
4essriesS of thLe cUentele and hacause the counselor must he^ 
decision purtog^e, ■ 

On ti^e other hand, the model of the physician, the social 
the engineer^ Jhe therapist, arifl particularly th^teachet are all 
jngful tnd affpfd us imporfant reterence points fcftr^ thf evq^utidn M 
th^ prof eWonal role of the counselor. Y^'Jn i^^rk with the disad- 
y antaged ^rticularly , counse^rs faced WifMlidLiitimbers of childffen 
musf evolve a unique role. The use of 
jfundamehtal to such an evolution, but it must 
,profession, not technocracy' Siagndi^, not 
not impartiality; effbrt, not expediency; and t 
systerns p| development, not reliance on those al^i 
In this sipriSQ, the poetinay have more to, say tha|i 
tific psy^htdlogist: I recall Thomas Gray's , ^ 

Full mMiy a gem of purest ray fer 
' ? ip- daric unf athon^ caves, bf^ orient 
' -" Full many a flower^L born t^^Bjish fli 

And waste its swetiSss, on'the deserKllr^ 



rit and science^ is 
, not'seientism; 

commitnientj 
^ance pf new 
available, 
ed sciert- 



Counseling, *of course, can 
Piaget (is correct in stating 
develop in the process ^learj 
structure^, with the morfi adv 
of the child's experience bnlyvif e; 
foundationar elements." ThiS js g 




i^vide the idylhciS^p^ertvi^oned;' 
en^ function^^df ^Pi^ipan ' beings ■ 
. they develop iAtbi^fiie^rcH|cal^^^ 
ms arising during the%|prse n^^^ 
'drift^ions hav^ pr^v^d'ttie 
ml, of rtieVbe^j kind, of 



0n: 




process in which we must .aclur(^^4 
provide a usefiilj though incidf^il, 
that .therp are 
iach ^ct of appli; 

ith the focu 
pie Improveme 
not in itself pro 



Vp ij^e f yenUoii , testing can 
tKT^K^we mdi^icnLbw' full well 
soci^li^^!^^d^',niorgt=resM inherent in 

bf a ij^^sitf^ineBi.de in dealiftg 

Jp'jc^sent drsb&ssio^f J,fc dMuv The sirSi 

''%^cience M^ technology 'of measurement, Can- " 
kind of rWults we must achieve.. - ^ l . 
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• jThi is hot the first time that the prob the measurOTeM.th^ ' ,^ 

*;*caUtuf«hy difrerant;hto engaged the. attentloil of p^id^ts m , 
' f Jnvi^tibhi'l Conf erehce- oh. testing Probleii;^. The .first ^papel » me , 
^-'iMr dottfttenw dtialt j with the topic; ^'Influence ff^lwralB^k- , ^ ^ 
' ground on Test S^erfbrpfianoe," and much of what was said atvtlftt time.| , 
f l eqi^f reU^ today. The secprid paw^ , , 

■ fb6asia flii':thtf W^^^ pevelopmmt of Unused" ^ 

'•TistsV' ai^d-alaini-what was said might- be repeated in Kjjjay s dfsci^. \ 
l:0^mimf^m^, 'f^^ sonic justilication. wonder ^^/hy•s»e•re,af it ' 

V ■ fheVr^l 6j the matter is, o,f course, that the topic is a hiihlf^m- 
bWbiie aiid deals wfh a central issue in the field of tasting, rtalhety, 
■slne^;itid»^durii differences art: a finjctioti of many f^cnir^'how dpe| 
i-oiifiiiisui^^af iri any factors are bfiip|meafc»^ ; ^. 
: ured' ftnd: irreievan* excluded?; Like the ; issU|^ ^^^^ 4 
objeodVe tbtlng'^^i which he'eds^to be exat^d,pe^|^cajly^^. 

to ihe fight qf" technical advttices in the field and 4^ spCid chants 
' tikirtg place in, thi context'^hi^ .which the issue is iriibeddeil; ■ 
' Atthe 19^9 Conference, TiinitfliU defined one position onjtK ^ue>^ ^ 



ih these iwprfd^:. ■ . * t ; . : . . , . ^ * .« 

lt:is ^^^Atendmi Hftt on .a predftUve test any^re ^iftpnce 
/b^^^^ grpups whose backgrounds differ should judgl^ not 
gdbd or.bad, nor right or wrong, but useful or not usefttl, yalid^ 
invalid for the predlctlqn of future behavior. (1 ) , ' . /# ^ .0^- 
i iOn the same panel Haggard stated the position which l^ be^ ex- 
pounded, perhaps with somewhat greater intensity but certaml 
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?^ M }7/no^^ej^tet clantyi teWng durmg^he jpast yeat/H^^ 

' % INl^ ^ri is Jj^at Cari iub*ciiltur#i 

^ip^Qti bbuhdaro^or IB it 0Dbu^;^^ '^Siye consider 

)€cii^e W luo jp|tiriTj^ i Juration ppportunities from tlie 
^ l^riy .gri^ how well a doiiS on our 

r' P^M^^ stands dizid teitis regir diess of; whether; iritt hd the0i 

^^^^^^ b^^ arfi oftfcn ig[vaii inftripr 

. 4 ^ ^^uc^tional opportunities j so that ^ vidous circle is s6t up^ andl a 

^^^^^\\'Jp0t I^tnbufU^ the issui; was whether or not diftere the^p^r-* 
>f fomancb of diflefent groups on a test are signlficahtly related to pre- 
V diction. If they are, then groups should be. treated separately when 
;';^makihg pridirtions, or the testijig ^ocedure should be rnodified to 
■ elinunate distorting differences: Blut nflfj let*s be sure that there is a 
problem, that is, thit the regression Urieg. for predicting a future per- 
fonnance are actually differerit. If they are the same, then how can 
one talk of bias? jPor Hapard — and ior many critics of testing today 
— flie issue is much broader than simply one of ftiaximiang predic- 
tion. Test scorts can be misused by uninformed people to perpetuate 
spciai injustiQes. Even where' the evidence indicat that sub-groups 
should be separated and treated differently in making predictions, the 
differentiations are often not made; and even if there is no evidence 
that regression lines are different — well, a prediction of the usual 
academic criterion may not be so important if by making it we elimi- 
l nate at the start those who might achieve in the long run. We may 
grant the predictor his point; the trouble is that his criterion is almost 
never an ultimate one, 

Essentially, the issue is one involving values; that's why it is so 
persistent and so resistant to the impact of research. Proponents of 
dfffeTent points of view tackle different kinds of problems and assemble 
different kinds of data. It takes*a long time for the collections to begin 
to converge, On the other hand, as one looks at the issue today in 
the light o^^hat can be learned from research completed and in pfog- 
resSj some converging is discernible. I believe we are ready to agree 
that a test which does not provide a valid prediction of some kind 
. of significant behavior is not worth the time it takes to answer the 
questjoni, At the same timej however^ I think we're ready to accept 




page 83' 



lte4 liivJJ«tloB«i Conferthc* on Ttsting Probltms 'V ' 

,■ J'' ■ :• " • \ ■ ^ ■ i* ■ ■ . 

.' , ' the proposiabn thit pute empirical prediction, irrespectfvIA Msslble, 
Indirect social effects, Is indefensible. Tests should be constru#cd an^ 
used with indirect consequences as well as predictl^ efIectivenM| 
. " in' mind. Finally, I thmk we've agreed that we hay^n educatiOftal 
problert on our hands. We can't concentrate all ou^ 

tjcatiOB of test users. My specific responsibility this aftemo^F^how- 
ever, is to discuss test construction, not the education of test users. 

'Hastorically, bbdi the College Entrance Examination Board and 
EducAtiqnfel- Toting Service have been aware of the si|nificance of - 
cultur^ W£ferehM5 , 'v^ the interpretation of test scores and have 
mdunted system® prbirams of research designed to clarify such 
dijfiEerencesI TOe differences recognized in Vm first Annual Report of 
,the Commission On Scholastic Aptliude Tests, 1926 are obvious ones: 

■ At some time in the future, then, it may be expected that difleren- 
tiaj weights will be applied to the tests for varying purposes. Boys 
and girls, engineering applicants,- apd liberal arts applicants would 
all take the same ixaminatlon under identicat conditions and their 
scores would, all be expressed on a scale in which the mean is 500 
and sigma 1)0, but the various tests determining the total score ^ 
would be differentially weighted depending on the sex of the appli- 
cant or the type of curriculum to be taken, (3) . ^ ^ 

Since that p'ioneer effort, literally thousands of studies have been car- 
ried out in which the Scholastic Aptitude Test or tests of the same 
type have been incorporated, along with other measures, in equations 
■for predictini the academic performance of specific groups. Member 
colleges and colleges contemplating membership in the Board have 
been encouraged^ to conduct studies in which sub.|roups that might 
be -expected to show different predictive relationships arc treated 
separately-' men and women, liberal uks and engineering students. 
Independent and public school applicants ,— logically, there is rio limit 
to the breakdowns one might try. There are, however, limitations of a 
practical nature. It may be that students from ihut little high school 
bver in the neighboring state, where all the teachers Jave .Ph.D.'s and 
. where there are a maximum of ten students* in a class iftight behave _ ^ 
. differently. The trouble is that a college ^ay get o#^wo a|phcants 
every other year frpm that school. And as for the ions of i^grant 
- farmers from Faraway County, wh'cr% there is one hMschlfl with , 
two teachers, neithe| of whom has yet completed the b^elor's degree, 
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the iituatloh is even less promiiing^ a college may have never had a . 
candidate before, and year U - 

It dpesn*t take much rniaginationito realize that if we expect each 
college to qondwl^ its own studies and develop separate predictive 
equilions for every slgnificantuiub-group of the applicant ^pulation, 
we^ being unrCT^^^^ iifnply faereq^ enough cases ^ ^ 

av^abie to establish a line of relationship, Inevitably, there comes a 
point where the trit mer is forced to make clinical judgments as he : , 
enGounters individual cases which seem not to fit into any of his^: 
previously, studied categories. This is not necessarily an Undesirable Vv' 
practices actually, it may be the only way to temper the Impact of 
'large-scale data processing methods on th^ unusual jndividual. . 
* Bnt'tetit poisible Uiat the be flduced sorttewl^^by 

. ^ direct attempt to eliminate irrelevant cultural d|perenc€s at the point 
O^Kt construction? And if the attempt is not entirely succes^l^ is 
^it not possible to at least identify potential problems for the teit Us|| 
I and pro^dde him with some guideUnes for his clinical judgments? THe 
College Board and lir? have thought so and have acted aceordirigly. - 
* With the developmiftt of large-scale data processing equipment, the 
^ question has naturally arisen whether or not it would be possible to 
pool data across a number of colleges and thus get around the problem 
of -too few caies in a sub-^roup. To date, the explorations of this ap- 
. /proach .have not been particularly fruitful; furthermore,^ the r^fine- 
meritsi which appear possible b^ separating for analysis special groups 
of^i^lMldates, are quite small ln contrast to the high*over-all validity 
ofJl^fe;test scores, sat scores appear to be equally valid for groups. 
ifi v^ith the average score is above 600 and for those in which the ^ 
averagi -score is below 400, in colleges enTolling youth from quite 
homogeneous backgrounds, and in those enrolling youth from a wide 
range of;backgrounds. This Is, to some extent at least, attributable to 
systematic efforts at the test construction stage to eliminate unneces- 
sa^ and irrelevant cullural factors from the test. 

To a considerable extent, efforts have been guided by rational con- 
siderations; but, particularly In recent years; systematic research has 
guided our efforts. Let me cite a few examples, The early forms of 
the SAT were made up of sub^tests, each containing so many questions 
that nobody could finish ir> the time allbtted: We knew from studies 
reported In the general literature of^psychol5^ that a fural youth Is 
likely to respond differently from the way an-urban youth will to a 
speed task. A^etherefor^iet about systematically to .reduce the weight 
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We kiitw that ipme parents could pay for special coaching in prepa- 
ration lor the tait whU^ tiot» and thai certainly the availa- 
bility of praQj^i^exe^^ vary widely* Thus, we decided to 
design Itenn which were relatively unresponsive to special coach' 
ing^^lperiigfe ; And we eollidted data indicating^ that mir (^ffoj^^Md^ 



been refiisonatil^ suqcessful (4) At the same time we prepared book- 
lets describing the test and prpvidiE^ enough illustratlw questions to 
insure candidates of the opportunity to know the kinds ot questions to 
expect (An order form for these free booklets is mailed to# every 
secondary school irt the country at the beginning ^ach school yfeir j. 

We knew that familiarity with the general content of a reading 
pass^e would iriiprove a candidate's chances of understandihg the 
passaife and answering questions about it. We also knew that the read- 
ing interests of candidates would determine to a considerable extent 
the kin4^ of 'materials they would read. Some would favor literacy 
reading, others sclentiflc; some would read agricultural magazines, 
others jwould read magazines aimed at the. homemaker, Our $tudies 
suggestedi for example, that girls were mpfe likely to do well with 
material in the biological^ and medical sciences while boys excelled 
with material in the physical sciences. (5) We therefore set up cate- 
gories to guide ouf selection of reading passages which would insure 
.a.hal^nce of CQntent. We. learned that women excelled in knowledge 
of words related to people while' men tended to excel with vocabulary 
related to things. (6) And we reasoned that regional and cuhural 
backgrounds would also be' reflected in the vocabularies ot diflf^rent 
candidates. We therefore established categeries to guide our selection 
of the verbal omnibus questions in the test. 

In order to increase our understanding of the cultural factors which 
influence test scores, we have in recent years deliberately sought out 
samples from^widely different backgrounds and studied their responses 
to questions. We have studied samples from Scotland and from Al- 
berta', from Nigeria and from East Africa, from the rural Midwest 
from New York City, and from segregated centers in the South. 
U our questions became more focused, we realized that we had 
methodological problems to solve. The central problem had been 
artticipated by Tukey in the disqussion of the papers at the 1953 Invi- 
itational Conference with this question r "Isn't the mainjnterest of 
this operation the comparison of these groups, the interaction between 
status and other variables?" (7) At that time, Eells and his associates 
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at die Unlverpi|y,of Chieago lidtfcoiiipleted studies in which the chief 
'Method Wis that of lom^ari the percentage of on& group marking 
a comet answer to a questi* with the percentage of another group 
making a correct ajasw^ to m same question. T^re were two limita- 
tions to this approa^. Itf the first place, since a' number of different 

qi}^r^ih^*^^f^^^^ thi ^amp pnpla, it was improper ;tg 

make the t€Sts of signlflcance as if the |uccesslve^ifferences were 
based on independent samples. The Replies to the dlftefent questions 
are necessarily correlated. Each new analysis did not yidd a new piece 
of information. In the second place, the effect one wishied to study — 
the inter^e^on betweerv,status and tes^questio^s — might ^ssibly be 
confounded with real abiliiy differences between the groups.' 

We have not yet solved all the problems of EOrtirig out the variSus 
factors contributing to diflerences in test performance of different 
poups, but we have made some progress and are continuing our 
studies. During the summer of \963, Cardall (8) applied the analxsls 
of variance deiigri for two factor experiments with repeated measures 
on one factor to data from three different groups and showed how 
mean effects' arid interaction effects might be separated, Sl)e demon- \ 
strated significant interaction effects for items in the sat across the 
three groups. At the same time, she pointed oiit that the statistical 
analysis did not permit us to differentldle between balance and bias, 
in a test^We, are continuing otir search for such a differen1iating _^ 
procedure. , \ - ^ 

In the summer 6t 1964, Fremer (9) examined Cardairs data sub-" 
jectively for evidence of balance or bias and concluded that whatever 
factors were causing the signiflcani interactions were hot readily ap- 
parent fjom an examination of Item conter^In a few instances, item 
content aould be related to reialive difficulty. For example, we plotted 
the item difficulties for a sample of candidates from New York City 

V against those for a sample of candidates from commiinities of less 
than 1 5,000 population in Ulinois and Indiana and fnore than fifty 
miles from Chicago and Fort Wayne. Only two of the 42 verbal tjues- 
fions fell outside a clearly defined ellipse. Both were easier for tlie 

• 'Midwastern sample and both involved gontent with a r\iral4layor. But 
by and large,, the factors producing changes in the relative difficulty 
of questions from group to group appeared to be the result of cornplex 
factors such as those producing individual differences within grtups. 
And since the int«ctions tend to be represented in the scatter plots 
by ellipses in whi^me points are distribute^ symmetrically, we have 
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^■ i&f^d to ^ in t6rmd of halan^ rather than bias. 

' When we hSre coinpa^ samples f rob outtide the Onited States 
, with- toose from ^ftin, we have found % wide variety of patttfrhs. 
EngHih speaking sffiiples^om S&tlana and Canada produce i^er- 
actions^o greater than th^ from different group^ within the United 
\^^^ h^ nprtajfi rype^ of math^aticaU aptitude guestions typicaUy 
^found In schDol textbooks it appears that the Scotch sample ^may have 
an advantap. W]^en ,we examine samples for whom English is a 
: second language and 'where differences in culture are immediately 
apparent, as with our African samples, however, both mean differ^ 
ences across all items and interaction, effects indicating chaages in the 
^relative difficulty of Items, arc Jramatie, Details of oufi finding^ have 
"^aen presented elsewhere: (\by Essentially, we found that there was 
^ a large diflferenf^ in over-alf perforroance and a large shift in relative 
diffic^ty of^questions, both attributable to the fact that English 'was b 
\ second language, tl^ shifts in reUtive difficulty were also attributable 
to the fact that ^e individuals in the sample had grqwn up in a rural. 
^ tropical society rather than in an industrialized country located in the 
North Temperate Zone, We reasoned, soundly, 1 believe, that m^ny of 
these differences were irrelevant to our purpose m assessing aptitude 
for undergraduate college studies in the United Strftes^ We therefore 
set about construcdng a special Scholastic Aptitude Test from which 
mappmpriate items had been excluded. Our evaluations of this effort 
are still in progress. Meanwhile, we. hiwe recognized that no amount 
of tinkering would produ^ a valid test on which African and Anjeri-' 
"can students of equal ability would make equal scores. T^e cultural 
differences are s^great that the Africans must ^ treated as a separate 
^ group throughout the selection and placement processv % 

For the most part, in^ assessing the validity of the test for African 
' candidates, we had to be^ content with the testimony of African edu- 
.cators==that their better students tended to make.Higher scores on 
tte test. We did locate one sample for whom both psat" scores, and 
Cambridge SchooJ Examination ma|ks were availablp and demon- 
strated ^at the^jwrrel^ions betw^^^total CamBridge ma^s and 
*PSAT-v and psat^m scores were simriar to correlations between school 
^^ades ^nd psat^ scores for samplcR ip- the United . States. Also, we 
found si^ pairs of African stud^ts for whom gr^es in an American 
college ware difecnlycomparabl^nceeach me 
the same^college was following tnb^amc cunriculum, Afsi^ test 
indicated a positi^ relationship between grades and p^i^m scores. 
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fv^n^tijh% systematic had been made to glace studpnji of 
abmty upi^e lame collegi. Presumablyi other tMngs being equal, 
the hipief the' icoi^, the more proihising the cancUdate. 

-ITie 4ita^m pur i^frican studies ipd m to become coneernfd 
aboyt pfter not a native 

lanpiap. ^tu^es fay Howell il l ) have providetf^vidence from whlcK^ 
we have developed a (rame of reference for iflterpreting scores of 
such candldatei, Thus, the bopfctet dpscribin^ tWe sat this year con-^ 
tains a section headed j "*If English is not your native languagei" 
which reads ' ; ^ ^ 

The Seholasyc. Aptitude Test is based on thfe; assumption that ttie 
.test-takir is a native speakar of .En^lsh. The *i^etbal guestipns on 
i^the teit haVfe been designed specifically to probe the kind of under- 
stancBte of4h© English lariguage that is acquired by dm who^Has' 
spo^^it all his life. TTie SATy in* other words^ tests not- so, much 
, your '*acadehiic vocabulary'' as you|' total mastery of the language* 
- This meanss of course, tiiat the iAT*is not entirely suitably for test- 
ing all toreign stiWents ; mariy of whom will take the teit : without 
/ having acquired this kind of understanding of En|lish- The extent 
of 'this handicap obviously 'will vary de^ftding upon ihe -degree of- 
familiarity with, ^d mastery of* English, / ^ ■ 
The College Board has taken steps tof^ake admissions offlceri ih 
American cplleg<g§ more aware of tjie language handicap some 
pudents"miiy have. If English is not your 'nati ve language, , and you^ 
are applying for admission TO an Anierici^ri that requires 

*the SAT, you should inform the ad(nisiWrii officer of 
of* the extent of -your, e^erience 'with English: Information of this 
kind will help him make a prdpfcr ihterpretatidn/^^ test 
Icbres. (12) ' - . * ^ 
* .^ur studies have shown generally that responses to. the; questions 
, in the mathematical sections of the sat are relatively unaffectedj;by 
differences in cultural background. Unlike the- veVbal aptitude score, , 
which reflects many aiiid varied influences within a culture, the, mathe- 
matics iJcill^and abilities axe'genemlly developed in schooK Such= 
differences as do appeft arc likely 4m bo reflectiolis of currictilar dif= 
ference^ across schools rather than li^nstic, geagraphic, or socio- 
e^^Oraic differences except as these ™y be indirectly^ reflated in 
school practices. We did find significant shifts in the relativ^ mfficulty 
of SAT-M questions when Ihey were answered by African samples. 
Apparently, there was much greater emphasis on routine compulation 
..in 'the African schools and' relatively less emphasis'on what we ^fer 
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to as "nover* items, "Aiid our indites of interattioh were, statistically 
significant although' relatively small when we compared samples from 
different .groups witW^ Stgtes. It is interesting, however, 

to note that CArdall reported interactictfis of the same magnitude for 
three strikingly different samples. Apparently, there's no greater 
difiference in the mathematical curricula bet weeh Negro schools in 
the Southeast and non'Segregatfed schools in the North thsn between 
schools in New York City and schools in the rural Midwest. 

The curricular developments in secondary schools at. thej present 
time are generating groups of "culturally* different*' candidWes so far 
as tests of academic achievement are cohcerned; and much of our 
research and development effort, during the past several years has 
been devoted to dealing with problems arising from this source^ These 
' efforts are simply the extension of a policy adopted by the College 
Board in the early forties when the Board moved from published 
syllabij and tests designed to cover them, to achievement tests desigped 
to measure the outcome of good instruction. This policy left wide 
flexibility for variations in details of courses from school to school. 

The problem of validation of an achievement test is somewhat 
different from that for an aptitude lest. While it is true that the College 
Board Achievement Tests have proved useful in pjedieting perform- 
ance in college, there are those who will argue, with justification, that 
the tests should be considered assessments rather than predictors. As 
an assessment, the validity of an achievement test rests on the judg- 
ments, of competent authorities that it requires of the candidate the 
performance of a representative sample of the' tasks toward which the 
instruction of the school is legitimately directed, * 

But how shall one decide who are competent authorities and what 
can be done about providing those authbritieS, once identified, with 
the data necessaiy to arrive at sound jiidgment? In the selection of 
committees of examinees, ici the developing of specifications for each 
new'tfest, in the .detailed test analysis prcpared'following the admin- 
istration of each new test, and in special research studies undertaken 
froi^ time to time, both ets arid the College Board are seeking to 
answer this question. There is only lime to mention a few examples of 
steps which are being taken to insure fairness to candidates with 
different curricular backgrounds, • ' . 

One thing we hafe done is to prbvidc committees of examiners with 
information about what is being taught in the schools. Such informa- 
tion often leads to changes in specifications for tests. T\\m, a study of 
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^ the preparatipn of mathematici' tantiidates^^^prpyided one basis fof* 
" ' ,\ dhangep in madiei^^ ichievement tests during., the current 

: & ^emicJ€ ar. (13) 
/ ^ mittees in matheiiia^cs imd in ptner fl^ds oyer the yeafs. ^ 
^ \ Ano^er contribution to the construction of* tests which ^mple^ 
. ' faifljt what js taught In the ichbofs has been systeinatlc compari^ns 
* *of the perfoimance of- candidates with' different curricular back- 
- grounds. TSeveral years ago* for example, we began oflfering a special 
examination for candidates who had' studied the pssc physics course 
^ * jtfter studies had indicated; that luch students , were at a diiadvantage 
on the regular Physics Test* At the same time i we set about designing 
a test which wbuld be appropriate for all groups of physics students. 
Now we are again offering a single, physics achieveinent test. 
J ' More recently, recognizing tha primary place of expert judgment in* 
determining the appropriateness of acHievernent tests, we have as- 
sembled and analyzed the judgments of teachers of various courses 
y " with respect to the/appropriateness of Questions in the College Board 
, tests. (14) ^ * ^ . * 

Finally^ we have undertalcen study of the relationship between 
p curricular background and tejt performance which involved all of the 
f ^ College Board Achievement Tests, ^ It is our hope 4hat the data from 
this study will provide us. with a vastly richer frame of reference for 
building ^pdciflcations to insure that t1i&*testH^ reflect accurately the 
learning of all candidates, . . 

I have wandered over a wide territory, and perhaps it would be >a 
good idea for xtm to summarize what I think Tve been saying about* 
principles of d^eloping tests for the culturally different, First, .since 
test scores are often misinterpreted or used in ways which reflect un- 
favorably on the culturally difTcrenl, whenever possible without seri- 
ous reductions in validity, differences should be eliminated at the point 
. of test construction<rSecond, if the lest is a predictor and if sub-groi^s 
. of sufficient size are available, the validity of a test should b^ deter- 
mined separately for each sub-group. Third, if cultural 'difTerenccs 
remain after careful test construction, the fact should be communi- 
cated to test users with guides to proper interpretation of test scores 
for those who arc culturally different. Finally, when assessments 
rather than predictioRs are involved, the basis of comparison should 
be expanded to include -the judgments of. different groups, of experts 
as well as differences in responses to test items by different groups 
of test-takers, 
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In the, course of prepajring to ■address this august Assemblage, I sought 
guidance from^&e Proceedings -oj the invitaiional Conference on 
Tesiing Problem fOT; the past fifteen ycsjs. This bibliographic search 
served two purpdses. FirstJ I. was interested in obtaining some esti- 
fnate of the lev^ of complexity of the papers so that I would not 
grossly underestimate the sophistication of the audience. (I had al- ; 
ready assumed that iE was; virtually impossible to overestimate the : 
intelligence of this groupj pecond, curious as td whether the. 

subject of measuring 'mvironments ha^'teen previously treated and, 
if so, t|> what extent, since t,didn't want to go o^er previously covered 
grouncj. ^ ^ . ' 

Mylsearch produceci the following resuUs: In 'the 1949 Invitational 
Confeilence, a, panel composed of Anastasi, Haggard, Stephenson and 
Turnb^ll addressed themselves to the problem of the; inftuence of 
cultural background oh test performance (6). The panel discussed at 
^^ength idiflferences in test perfor^nanqe fof various cultural groups and 
considered the problem qf test jtcms which seemed ;£o differentiate 
varioui cultural groups. The reports of the panel membejs as well as ' 
the lively discussion that ensued have fortunately been preserved in 
the Proceedings. In isj56; Bayley delivered a paper on '-A New Look 
at the! Curve of Intelligence," which contained a section on, some 
enviroiimenta;! determiners of intelligence (2^). Unfortunately, a time 
limit pfevented Bayley; from developing this section of her paper more 
fullyr^inqe 1956,, the Record indicates that the topic of environmental 
measurement has not been'dealt with. This seems to me to be rather 
UriWrtuhate, arid ^shall endeavor to draw your attention to^ this rather 
neglectjed area. • _ ^ 
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T]here is abundant evidencje 
we find m test perform^ce a; 
' gnVirQhpdenits in- whkh i^d^l 

V stotid rither. spectacularly 

- and most recently in the . work 
Hunt (9) -arid 

/nutnber of sUchVrtudies, Howev^ 
' %ence, : we have oersisted in our vie^^ 

: vidu^jistic tertns^^bjft is, the Individual iS 




|he bbhayiorial yifferences 
^sult gf differences in the 
rather ^han of inherent 
inain|^ltnr^eBn*^dei^ 
(13) and; Kirk 
[glass in England (5)\ 
rized the results of a 
accumulation of evi-" 
iing in essentially incji- 
snsidcrcd an "X/' to, 



be solved, aad the envirqrimfent in wh^k the ifflividual ^s lived is 
considered as bjit one iource ol err^^hich can be handled by 
appropriate wei^tlng cA^est^snlts .£^me other iprocedure. This 
viev^ of^testing is renec^ in JkirosrM^fc Measurehents Yearbook, 
Which contains descriptions an^Tcyi instrumehtS: 
for the measurement ot in^^Iffial chur^teristics, bu^^^yery few for 
theWasure^ent OT envirbnments, . ; \, 

This is indeed an unfortunate SJagiof affairs. fill theories of learn- 
ing and behavior ^ake, provision foTthp Influence of the environment 
on the developm&nt^f human clia/acterishcs but,^s noted above, we 
' have' not had a correspoii^ing cfttiphasis in ©ur measuremem pro- 
.cedures. I would a^o submit that We have rarely ajtOTpted to sys- 
tematically rdjfe mdividual test* data to enyifOnmcQ^^^ta in ways 
which^ are denned to imirease. bur underHtanding of 'ffie interactive 
process between the intiiViduaL and the cnvironmenL ' ' 

To be sure, we do ha^e'iqmc environmental measures. However, 
the number of in|truments designed toimeasure characteristics of the 
/envlronmfenf is quite smalP when compared with- the number of in^ 
struments deliipfed^to measufe characierislicK of the indlviduaL Also, 
those en virbntrlrital measures that arc available are usually limited 
to general measures of social slatiis or economic well-being. However,^ 
if wduldleein that just as a general mfcasurc of intell^nce or IQIias^ 
obscured many important differences ar^ng individuals, so a general S 
Tndex of social status pr economic wcll-^elng has: obscured many very 
imporlant, differences among environments. The work of Kahl,(l(T), 
for ejcampfe, has been quite reveali^ in Us findings of some of the 
differences which are to be found \mhin a given social class. There is. 
hoN*ever, an even more basic dimculty in the use of general index 
of social status or economic well-being to characterite^m environment. 
Such indices usually represent a summation of a number of sympibms 
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Or surEaqe'cl^Eil^te^^ io! jari; environqjttnt aftd^^ as suchilgiVQ little . 
.f^onnk^n a^ut/the spediflc ways ^ ^V^dh* env^ronmenfal/^f^ 
:6might^^ffa* ^therde wto p^ char^cteristici. 



J/ tliis point ^bphBiB^il^ Bayl^ in her 1M§ address: and was 
^ one of ^ jC^^^^^m^B in -AriastasPi/pap^r, *-l|^r^^ Enyirpnment 
: Bn4fim0Ji^^ \''^\/'''^-\- ,: ■ ' 

^ ^ iff our v^Hij we ha^^ attempted; to iqJlo\fr $om of the guidelines 
for envirdmeDity -research suggested by Anastasij Bay^e^, Bloom, 
; and ottiers; la d0ing so, j^e havi tiiilized a coJiceptiori of the eriyiron=, 
// 'ftient ^hldhii d[|ff0r$M fjfs^M lhat uspd.in thtf d^^ 

envii0nnienlal ^iieasiJ^s|ih fotir way ^ . . - j : ; 

1 . Instead of viev|l^ing an environment .1% a single gntity;' we- have ; 
po^liulated that fa sirtglf ^h^isfcal environmerit inay bfr niade up af a 
/ nunibif of su^-enviroDnients, with each sub-environrtent bparatirig 
' to influence the develppmiht of a specific charactoristic; Eafr purposes 
; of measurement, we would coifcdlve of an .|nvirontnent fo^ the devel- ^ 
opment of itature, another environment for the developnient of gen- 
eral Inttelligencej another for the development of in^pendence, and 
* so forth. ThuSj for usv the problem of measurftig an environment was 
reduced to the iidantification and measurement of those. aspeets pf ^he 
total enVlronmsnt whiqh were likely to be; related to the development 
of selected specific characteristics, , 

,2. Rather than measuring some of the more surface manifestatiphs 
or symptoms of an environment, we have been interested in measuring, 
environmental variables which we have hypothesized would be likely 
to directly influence the development of specific characteristics, This 
dejparture from some ^f the more conventional a^prdaches has, in the 
case of m^suHng thef ihome environment; for example, resulted in our 
investigating what parents do in their interactions with their children 
rather than what parents are in terms of status level of father's occu- 
pation,, type of dwelling,; source of income, and so forth, 
«^ 3. We have attempted, to summarize and treat environmental data 
through the use of psychgpietric procedures^ Insfcad of isolatir^ anj 
particular environmental variable whichr miglit b,e related t© a par-r 
ticular characteristic, we have attempted to sampl§ a variety of the 
* proc^es and conditions in an environment wrtich were hypothesized 
to be related to a characteristic, and to summarize- these in d^cHbing 
an environiiriient. This Is essentiallj^ the sanie procedure we follow in 
testing. That is, we would.no sooner describe a student's performance' 
in ^ certain subject by his r^sp»nse to a single test item than we would 
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'describe an environmettt on the basis oUa. sin|Ie, feature of that en- 
; • viroliihent. We tlave felt, in other words, that a sununarization of a 

• ndmbfa nf wiaWes m m iritgortan ^ ■ describing, an^ e^^^ 
. ' a number of tSst items is in, describing a st^^^ » 

; subject. V • . r „ 

4. We have attempted to.systematically relate measurements oi trie , 
' ert^iranment to measurements of the individual. This has been an ,^ 
'e/tremely im^rtant feature of our work. A hummer «f investig|tor| 
Have developed enVironn»ental instruments \yhioh attempt to measure 
; elected aspects of art environment, but have not j-elated environmental 
data to individual' test data. In our wortc. the importande ,of sys- 
tematically relating .the two kiBds of data can be indicated by noting i 
' ; that one of tl« criteria for thfe validation of our environmental meas- „ 
jires Has been individual meaiHres of the characteristic under study. 

T6 briefly summarize, our approach to the measurement of en- 
vironments has. been characterized by^conceiving of specific environ- j 
. ments for thj development of particular characteristics^ attemptmg ,^ 
tb measurtf-^vlronmental variables which ware hypothesized to be 
directly felated to the development of partioular character&tics, sum- ^ 

: , .marizing ail^ treating environmental data 

metric procedures, and relating environmental measureii^s to indi-- 
vidual measurements. The remainder of this paper .wilt be devoted to 

describing the development of two envitonmental mpasures based on 
. the above ideas aqd presenting some of the results we hpye obtained. 
While environmental influence^ may range from' the most immedi-^ • 
ate social interaction to the most remote' cultural forces, we chos 
the home.as the physical environment for our study,. The reason for _ 
. "^this was our assunjption that the home produces the first, and par- 
haps most insistent, influences cjn the development of tHe charac- 
terisUcs we were cdncemed with — general intelligence, and academic 

achievement, , , ' 

The Jim step in dcyeloping our environmental measures was to /' 

compile a list of the conditions and processes in thg^ehvironmentj^hich 

theory and previous researdh' indicated were llMy to influ|i|e the r 

/ development of general intelligence and/or aakpmic achi#fSment. 

The areas, from which such variables were dra^n included child de-* 

. velopment, , learning theory, motivation, and Jpsychometry. Special 

Efforts, were ma^ to select variables which were as closq as possible. - , 

' ■ <*to being envilfonmental 'counterparts of the components of the indi- ..' 

, . ^vidual characteristics. The en^ironinental variables which •were iden- 
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tified as llkqjy to be related to aGadenli^chieyerneht were*;/ihe climate 
created for achievement motivation; the oppgirtunities provided 'for 
verbal developniA^t; the nature and apiount of assistance provided in 
' oY&rW \ th^ activity , level of^the signific ant 

Individuals in the enVlronm||rtf the Tevel of intellectual ityMn the en- 
^ vironment; arid the kinds of work habits that are expected of the indi- 
viduaL Environmental variaBles identijBed as .likely to W related to 
general . intelli|ence ^were : the stimulation pro^idfcd for intellectual 
growth^ the opportunities provided for/ and emphasis on, verbal de- 
velopment; and the provision for general types jOf leaming in a variety 
of situations/ ^ , I ^ * 

No claim Is ma^^that^hese tvfo lists of variables exhaust the range * 
* of, conditions in the-' home which can iri flue nee the development, of 
academic achievement and general intelllgwce. Rather, they ai^ 
viewed as a sample of environrhental conditions which were hypothe^^ 
sized to be related to the developtAnt of these characteristics. 
^ There arc two notabl^ featur^ about these two lists of mviron- 
\m€ntp\ variables. First, there jy^ considerable overlap between the 
Wo lists, whichMs largaly a reflection of the overlap between |the two 
* characteristics, as ^s beeif pointed dlit by Kelley ( \ 1 ) ,and Colemdn 
^ and CuretOh ,(4) . Second, the variiibles were stated in quite general 
terms. This l^cW>f spegificity made It necessary 40 define each environ- 
mental variMb^' i^an operationahfogii for purposes of measuremf nt< 
^ 'Thus, for each variable, we develope^a of-prbcess characteristics 
^' efonsisting of Specific behaviors of parejits and othbrs in. the home 
, whfch were likely to be related to genera! intefligcHce and/or aca- 
demic achievement. , ^ , \ ^ ' 

An illustration or two is perh^apsTn order One qf the variables 
hypQthe^zed' to" influence the developmgpt ^or academic achievement 
, syas^hj^limate in the home for achiavemfent m^j^ation. The specific 
process characteristics which were defined, as comprising this varit^e 
were: the parental aspirations fo? the child's, education the parents' 
iwn aspirations; ^ parental concern /^r academic achievement; th| 4 
0 sbcial press in the" home^^oii^caden^achi^ement; the rewardLlrc- 
■ coriJed academic accomplishrnents.; parental knowledge of the epucf- 
tional progress of the child; and the preparations made for the attaln- 
^*merit ,Qf educational goals. The breakdown of each environmental 
^variable jnto specific measurable characteristics was^ in each case, 
^ based ok relevant theorj^Jpri previous res'ea/;ch, In^ similar way^^he 
variable termed "'opportunities" which was provirfcd for Verbal ^de- » 
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^i^opAm^^om of the three variables hypothesized as con^pnsing 
y ' the inviroiment ior Uie developqpent of |eneraK IntelUgpnce was 
\_anaiyEedt Int^ to§ tollowing pmcess characteristics; -the einphasii^n 
• ot lavage ifa a Wyety of iituatlbi^^ upputiunities-pro^id 

for eEdirging vocabulary; parental emppsiii%n corrtclnpss ^f us^e; 
qudlty of la 
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^i^ tfie poeess eSracteristlcs cpmprising each fnvironmentai ^ 
varia^wera ^cii^ed; it wis necessary to dj^elop a sef of procedure^ ^ 
for gathering evidence about the strength/ op^each characteristic . The • - 
prootdtfr^. flpaUy setlie^ invplved the use of^an extended inter- 
view wiSi t^e mother (and sometiities both pajiffits) fti the Jiome wither 
Oilt the child being present, and the use of a Set of j»Ung sc^ales, Ques^ 
tions were .developed to elicit ^formatbn lbou4 t^e presence and \ ■ 
strength of each proems characteristfc. These question^^ere then : ' 
^ganized^into an interview schedule. At^thq^sam^ ti^^a series of # 
mmj icMes^as constructed: Thesf scales were used to obtain ratings 
fbrrfachbf me process characteristiQS. 

Once thelnterview schedtile was developed, it was pilot-tested tns _ 



y a small sample ot homes. On the bisis of prelimihary findings, the 
f interview schedule wa# revised and )[^tested. This time it was foun^ 
* ^ that eriou|h infomation could be obtained in an inttirview lasting 
ayoift an hour and a half and consisting ofi03 questions. The >nter 
* . mation that was obtained was u^^d in raUng the U process char%- ^ ^ 
teristics' comprising the ifjstrumenrimaasuring m environment for 
general intelligence, and Ahe 21 pr^ess characteristics which com« * 
, prised the instrumenf -measuring -^e envir^ment for academic^^ 
achievement. 

It was noted eariier tliat 4he focus of our investigation was what^ 
parents did rather th«tewhat parents were in term^ofltatus, economic ^ - 
^ welUbeing, or some other .demographic ^rible. The questions in the. ' 
Interview schedule were carlfplly designed to^lipit informatiQn about 
what parents actually^id insofar as general in^lligence and academic 
achievement were cohcemed. ^\ ' " , ^ 

One process characteristic' which was measured,' for example, \ 
the parents' educational aspirations for the child. In the course of ^ 
interview, the parents were asked howlniuch education they wishe^ 
their child to have, how much cducatiofr4hey actually expected their ^ 
.child to receive/and the minimum iimoUnt of education thej' felt 
^heir child must have, .In response to these questions, a number of ^ 
parents indicated that tlfty hopc^ their child would receive a college 
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education, Later in the intei-vlfiw, the topic was brought up again, and 
parents who h%d indicated that they hoped their child, would go on to 
eoUeg were queriedtas to ^hat plans had been nladc to finance a 
college edJbati©Ti,/T^*^i(|ft^ to this latter question were most, 
illuminating m differentiating, a seemingly homogeneous group — par- 
ents who hop^d that their child wbuld have a college eHffiation. The 
responses rangerf from the Jotal absence of any plan for financing a 
colfege education to the most elaborate of plans. At the high end of 
the scale were several parents who ha4 already established trifit funds 
earmarked for their children's college expens^. It may be noted that 
parents who rated high on the planning jfor the attainment of educa- 
tional goals were also the ones who indicated that education was fre- ■ 
quently discussed in the home^ and that tSfe^ child was aware of the 
educational ^lans that had been made, ^ ^ ^ ^ f 

In the main study we se?ected a sample of 60 homes frbm a medium- • 
si^d Midwestern community comprised of urban, suburban, and 
rural areas. Preliminary info^atiQn was collected on all 1 ,062 fifth 
grade Itudents in the school system. On the basis of student descrip- 
tions of the fathers occupation, social class ratings were obtained and 
a stratified random sample df Romes was selected for study. The num- 
ber of cases drawn from each social class grou^ftg was proportional 
to the number of male adults in each grouping in the*U,Sr^opulation 
according tQ Department of Labor data, ^ « 

Contact was mUde with each of the selected homes aijd an inter- ^ 
view wifh the mother was requtsted. In those cases. in which the ^ 
mother was unwilling or .unable to participate, an alternate home was 
selected. The interviews were conducted in the home with the mother 
when the cftild was not present. Td^ establish rapport, the mothers were t 
assured at the outsot of the interview that all informa^on would be 
treated confidentially Und that the school would receive only group 
resu^js and not individual reports. . \ ^ 

On th^ basis^of the information collec^tcd during the course of the 
intervlewsj eaehTiome was.rated on each of the process characteristics - 
comprUing the two instruments. For each environmental measure, all 
homes ^fefe fated on one process characteristic, then on another, and 
"so forth. The process characteristics in^cach instrument were averaged 
to obtain. the environmental variable ratings. These ratings were, in 
turn, combined to obtain the two total environmental ratings — one 
for tK environnlent for academic" achievement and one for the en- 
vironment for inte^li^ence/Thus, it was possible to conduct two sepa- . . 
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rata inveiUgations based on the same inter^^^ * 4 

At a later time, achievement test icgres on the Metropolitan 
■ Aahiftvemen t TeitBaygQ^ d iritenipnc€ test scores on the Hen mon- 
^Nelion Test of Mental Ability were obtdned from school files for the 
^60 fifto graders in the homes selected for study. The environmental 
data and the test Ata were then systemalicalty related^ / 5 

A fuU discussion of the results of our work is unfortunately beyond 
the bounds of thji paper. However, the major findings can be pre- 
s#nted and some implications for further researdh as well as educa^ 
V tional practice can brtentatively drawn. The first major finding con- 
cerns the relationship i^tween the total rating for the intellectual 
environment and measured general Intelligence. The correlation be- 
tween these two variables was 4.69. This can be oontras^d with the 
correlation between socidpffttus and measured general intelligence 
which has been found to he between +.20 and +.40. Accepting the ^ 
correlation of +.40 as the correct estimate of the relationship between 
social status and intelligehee, It would seem thatt the newer approach 
to the measurement of the environment accounts for about three times 
as much of the variance general intelligence as a me«^ure of social 
status. ' , . 

The second major ^finding in our work concerns the relationship 
between tfie total rating of the environment for the development of 
academic achievement *d achievement test data. The correlation be- 
tween the total environmental rating and the total achievement battery 
score was +.80. Again, this may be contrasted with the correlation 
between social status and academic achievement which has been found 
to be of the order of/+.50^ In terms of the proportion of variance 
accounted for, the newer approaclf^to the measurement of the en- ^ 
vironment would seem to account for at least two and one half times 
as much of the variance in total academic achievement as a measure 
of social status. Taken together, the two corrcjations between the new 
environmental measures and intelligence an3 academic gghicvement ' 
reflect a new level/pf relationship between measures of{?thc environ- 
ment and measures of Individual characteristics. 

This new order of relatipmhi|p ^n have a number of important 
implications for theory, research, and practice. The fyidings would 
suggest that the conception of a single physical environment as con= ^ 
sistlng of a .number of sub^environments for the development and 
/ malntenaricefof specific characteristics is a powerful one indeed and, 
If fully dev^Ibped, could greatly enhance ou/ understanding of the 

page 100 * ' tf 




* 



.. m 



^ Richard Wolf 



♦ interactive process b^ween the individual and the envrroDLment - 

' ■ The cofrelations between the ovifer-all envlronmeHtal rating and 
Sfiegific acMeyement ^eores ranjed from 4^ .S5 for arithmietic 

computation to =f .77/for word knowledge. , The sis' Eub4ests^ r|^^^ 
in decreasing Order of »their correlation wfth the over-all environmental 
ratthg w^rer word knowledg^; reading; language ;.ajri^nfietic pfoblem 

^ solving tod cbnc0pts; word Hiscrirtiination; and arithmetic fcomputa- 
' tloft. This ordering of the various sub-test score correlafibn^ with tiie 
^•total environmental rating might well have been anticipated. The in- 
flu^nce of th_6 environment on ac^emic a^cvement could be expected 
to be* greatest in- the language are^ because much of the basic socializa- 
tion of the dhild is usually accomplished through the medium of lan- 
guage and^ in 6ur culture^ tfeis responsibility rests prptty clearly in 
the^^omp. Thus, the, fact thst the three highest correlations with flie 
totatenvironmental^ting were word knowledge, reading, and language 
might well have been expected. Similarly^ the. one individual charac- 
teristic in which the responsibnity for deveTopment Wsts most clearly 
with the school is*arithmetit computation,' Again, the fact that arith- 
.metic computation^ was the lowest eorrelation with the over-all ^en- 
vironmental rating might also have been ahticipated/ It should be ' 
noted that this ordering of the correlations between the achievenient 

- subtest scores and 4he total environttiental rating helps to establish' 
the construct validity of the two instruments as much as do the 
correlations with over-all academic achievement and with general \ 
intelligence. 

It was mentioned eBLrlier that the correlation between the over-all 
rating for the environment for the development of academic achieve- 
ment and the total achievement batter)^ score was ^ + .80' This repre- 
sents a high level of relationship and can be considered to be quite 
sufficient for purposes of prediction. That is, a measure* of what par- 
ent^i do in the home can be used to predict school achievement with 
a fairly high degree ot accuracy. In actual practice, however, the results „ 
of intelligence tests are often used as the basis for making decisions 
about the placement and educational treatment of'students. This fact^ 
■ prompted us to find out what^happehed when intielligence and environ- 
mental data were combined to predict a^SHdemic achievement. This 
involved the computation of the multiple correlation between IQ and 
4he over-all environmental rating for academic .achievement with the 
total achievement battery score. The multiple correlation thus obtained ' 
was +.87, This represents an extremely high level of relationship. In 
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fact, this is alrnQst the upper limit foXsuch 9 correlation when one 
takes into consideration the reliability ofthc various iijgtniments. Sta- 
tistically speaking. 'the amount -of *var^nie in academic aohieyemeiit . 
accounted for by intelli|»nce test scores alone was 58 pep cent (r ^ 
■*4,,76). Whett a measure of the environment is added, the amount of, 
' variance in academic' achievement accounted fat rises to ,76 per cent. 
Thus, the addition of a measure of the eavironment greatly enhances 
the estimation of acadwnic Achievement. 

■ Since the studies described here represent^ the application, of a 
psychom'^tric approach to the measurement of environments, it is cus- 
'"tomary to report some tec'hnical drta about the instruments. The , 
.reliabilities of the two. -instruments wwe estimate'd through' an analysis 
♦•of variance, proccduje sUggcstea- by Hoyt (8), The reHability esti- 
mates obtained thrqugh-use of this procedure were .89 for the measure 
of the" environment for the development of intelligence and SS for 
the measure of the environment for the development of academic 

achiev^ent. , ^ 1 

Valiffity data about the two environmental measures mheres lar^ly 
in the corrections with the i^asured individual characteristics — gen- 
eral intelligence and academic achievement. Additional evidence about 
'the validity of the measure of the intellectual dtvtfDTWfent was ob- 
tained/through a double cross-validation (it's imprtant to rfimember 
that the hyphen comes between cross and validation). This procedure 
' was proposed by Mosier (.14) and is designed to furnish evidence 
about the stability of results based on a single testing. It involves the 
random division of -the total sample into two sub-samples, computa- 
tion of regression weights for each sub-sample j^arately, and the 
application of each'set of regression weights to the other sub-sample 
for predicUoiTof the criterion. In our study, the resulting correlations' 
for the two sub-samples w.ere +.663 and +.664. These are extremely 
close to the correlation of +.690 for the total sample and, as such, 
tend to fortify the original findings. 

There are a number of additional findings but, unfortunately, these 
will have to wait umil a fuller report of our work is prepared. How- 
ever with even the sketchy outline presented, some generalizations 
and implications can be drawn. The first generalization we would offer 
is that the conception of the i^ironmcnt upon which the present 
instrumeijts were based does indeed seem to be a fruitful one for meas- 
uring and analyzing environi^cnts. The high correlations between the 
environmental measures anU the specific characteristics selected for 
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Study furnisK' direct suppbrt for this generali^tion. However, we do 
*not regard our list of environmental variables ^s b^any means com- 
' ■ plate, Hopefully, further research will result in the identification of 
other environmental variables, ^hich can be added to*, or even replace, 
- ' some of our variables. This will have to be determined at jome future 
date. The point that we would emphasiz^is that research on the meas- 
urement of environmenfs be directed towards the identification and 
measurement of. ongoing environmental processes in relation to spe- 
cific human characteristics. It would seem to us that environments for 
, , / the development and maintenance of such characteristics as depend- 

- ence, aggression, dogmatism, and others could be delineated and 
measured and systematically related to measures of the particular 
characteristic. Such research endeavors could greatly enhance our 
understanding of the development of many, human characteristics. 

A second generalization which emerges from our work concerns 
the relationship between the environment and intelligence and aca- 
demic achievement. Much of our. work gives quantitative support to 
the ideas of the effect of experience on intelligence and achievement 
put forth by Hebb (7), Hunt (9) and othirs. The level of relation^ 
ship between the measures of the environment, and intelligence and 
achievement are significanlly higher than those found in previous 
siudies which used sBciaPstatus or some other general indW as a 
measure of the nature and quality of the environment. Moreover, these 
Hew levels of relationship seem to be quite in line with the expecta- 
tions of the theoreticians. It is again hoped that further research will 
be undertaken to test these findings. 

If additional research does confirm these findings, then it Would 

^- seem that there are a number of implications for educational practice. 
One obvious implicatiDn would involve the development of new cur- 
ricula designed to help overcome identified environmental deficiencies 
among students. Useful information about the ingredients for pro= 
grams of compensatory educLiiion could be obtained from careful ex- 
amination of the environmental measures. The reason for this is that 
the environmental process variables identified for study are, by and 
large, educationally malleable ones. That is, the variables represent 
. environmental processes which can be manipulated in an educational 
situ^ion. For 'example, one of the process characteristics in the meas- 
ure of the Intellectual environment was the opportunity provided for 
learning outside the home (excluding the school). If the homes which 
students come from areiound to ^rate low on this characteristic, it 
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would seem that the school might well undertake td furnish a number 
of experiences in this area. An extensive program of field tnps, for -.^^ 
example, could be undertaken, to overcome this deficiency. Similarly, # 
if the homes which students come from are found to be deficient ;;n <^ 
according rewards for academic achievement, the school^ could under- . 
take to develop a greater system^ of rewards for S4ch accompUshpients 
and/perhaps, begin to work in concert with the,^me in insuring that 
academic achievement is acknowledged and rewarded. The point 
here is that the measure of environmental processes rather than status 
characteristics can furnish information which can be of direct service 
in the planning and conduct of educational programs. 

It is rather fortunate that the fr^ework for educational,^rograms 
designed to overcome environmeflTal deficiencies already exists at the 
elementary school level The existence of self-contained classrooms 
and the recent development of an ungraded primary system could 
greatly facilitate compensatory educational programs. Even the tra- 
ditional parent-teacher conference can be of great assistance in edu- 
cational programs designed to overcome environmental deficiencies. 
As currently used, the parent-teacher conference serves mainly to 
Inform parents pf the chiid^s progress in school It seems possible that 
this function eould be easily expanded to include the collection of 
certain standard information about the home environment jTmjs, en- 
vironmental data could be easily gathered and serve as a basis for 
planning programs of instruction. 

There are a number of other practical uses for environmental data 
which could be suggested. However, the mfest tantalizing problems are 
probably the theoreticlil ogeSi For example, how stable is^the environ-^ . 
ment or, ^l0re importantly, what arc the conditions which make for 
stability in an environment? Also, how. pan one assess the variety of 
environments affecting the child as he. grows older? What are the 
points when environmental interventidW; will have the greatest likeli= 
hood for success? These are thorny problems, and we do not have the 
answers. However, we do believe we have a methodology which can 
be of service in attacking these questions. Essentially, this methodology 
involves the application of our measurement procedures to the study of 
the environment and consists of four steps. 

The first step entails the definition of an environment for the de- 
velopment of m specific characteristic. This construct is drawn .largely 
from the relevant theory concerning the development of a particular 
characteristic. The second step involves the identification of the spe- 
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cific environmental conditions and processes which are Ukely to di- 
rectiy affect 1h& development of the characteristic. Again, this step 
involves the extensive use of theory coupled with previous research 
concerning the development^ of Jhe characteristic under study. The 
third step consists ^ the collectron of evidence about the various en- 
virorf^ntal processes. In our work, we made use of an interview-and 
a series of rating scales. However, there are a number of additionaL 
ways to gather such evidence. The work of Pace-and Stern (15) sug- 
gests that the use of participant observers can furnish important Infor- 
mation about an environment. Observational procedures can also be 
utilized. It would seem that m&ny of the procedures that 'have b'een 
developed to measure individual characteristics can be adapted and 
applied to the measurement of environments. The fourth step in meas- 
uring an environment consists of treating environmental data through 
the use of psychometric procedures and systematically relating data 
about the individual to data about the cnviron^ment. This is an ex= 
tremely important step since it Is only through this process that en- 
vironmental Information can give full meaning to data about indi- 
vidual char|cteMstics. 

In addition tq the four steps listed above, the same technical con- 
siderations abouji reliability and validity which we are concerned with 
in the development ofc tests of individual characlerisucs must be dealt 
with Again, our psychometric procedures can be utilized to gam 
answers to these technical questions. 

It is not necessary to folloV the four steps in the order listed above 
and study the environment in relation to one characteristic, then an- 
otber, and so forth, A number of environments and individual char- 
acteristics can and, perhaps should, be studied simultaneously. It 
would ^eem that factor analysis, which has proven so powerful in the 
identification of the major dimensions on which individuals differ, may 
prove to be as powerful in defining the dimensions on which environ- 
mf nts differ, 

in closing, I invite you to give.,consideration to the prospects for 
environmental research. I would submit that as we make use of the 
conception that a single physical environment may consist of a num- 
ber of sub=environments and seek to develop measurement procedures 
that capture the operant social and psychological conditions and proc^ 
esses in an environtnent, we should be able to greatly increase our 
understanding of the process of how individual characteristics are 
developed and maintained. Only when we have such unHcfstanding 
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c& we develop more adequate theories of behavior, and only then 
can we determine how particular cliaracteristics-can be maintained 
or altered. % 
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TTiis paper is a Inecessarily. abbreviated report of a study completed 
at Hunter CoUege under the^cdrdirectorship of Professor Gerald 
Lesser, Dr. Donald Clerk, and the author, in an attempt to cover all 
the most interesting Hjiects of the study, I may'occasionally be too 
brief. HcKwever,^ the full report will be avaitablc within the next few 
weeks. 

ITie problems pf differentiated mental abilities and of their rela- 
tionships, to class and cultural group composition challenge us directly 
in the .urban centers. The provision of suitable educational programs 
for all of the children in our schools presupposes our knowiedge of 
what these chiidren are like. ' ^ 

Despite the considerable amount of work by psychologists in the 
field of mental abilities to^^ create so-called culturc^frec or culture-fair 
tests, little has been shown, to yield consistent and valid results. The 
problem still remains as to how to evaluate the intellectual potential 
of children whose .backgrounds necessarily handicap them seriously 
on the usual tests of mental ability. 

This study foctised on two major aspects of the problem: first, to 
devise tests which wpuld be as free as possible of any direct class or 
cultural bias btit also would still be acceptable measures of intellectuqj 
traits. (It should be noted that no attempt was made to devise any 
remarkably new or unique testsi) The second goal was to structure a 
testing situation which would enable each child to be evaluated under 
optimal conditions. * ^ ^ 

♦This sludy was made possible by a Cooperalive Research- Grant from the 
, Office of Education^ United Slates Deparimenf of Healthy Education atid 
Welfare. . i ^ 
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The specific go^ of this 5tudy was^ to examine the ^patterns among 
v^rious^mental abilities in first-grade children ftom different social 
class and cuftiral backgrounds. We accepted a ^definiUon of intelli-^ 
gence that postulates Sv^m mental abilities; hence, intelligent be- 
havior can be manifested in a wide yariety of fonns. This providjil , 
the premise for the:hypotheses tested in Che study: that class, arid cuU 
tural influences difer not fenly in degree but in >ihd, with the conse- 
quence^ ^that different ^inSs of intellectual s]|^lW are fostered in variv 
ous environments, . - _ ^ ' " t ( 

The s'pecirfc hypotheses tested were thdt; . , 

1, significant differences. texist ampng grbups of children from dif- ^ 
ferent 'social class and quUural backgMnds in each of the mental 
ability areas specified below; ; | 

2:' significant difTerences exist amonf'groups of chndren from dif- 
ferent social class and cultural backgrounds in the, pattern or con- 
figuration ti^scores from these diverse. areas of mental ability; 

3. -significant Interactions exist between the variables of social class 
and eultural background in detertnining the level of each mental abil- 
ity and the nature of the patterns amdng them. 

The four ethnic groups selected/for the study were Chinese/ Jews, 
Negroes and Puerto Ricans. The design provided for 20 bpys and 20 
girls at both the middk class and lower class levels in each of the 
ethnic groups.^ The total number of subjects was, therefore, 320, The 
age level selected was from 6 years to 7 years of age,, The actual 
ultimate sample ranged in ap from 6 years 3 months, to ^7 years 5 
months, the sample was obtained with a few exceptions entirely in 
, urban, congested New York City, In brief, the .sampling procedure 
' consisted of a rather' detailed study of census-tract information on 
ethnic group distributions and income-level distributions in the city. 
Additional data were obtained from a variety of agencies including 
charitable institutions, government bureaus and market research com- 
panies. After careful plotting of the location of our population, the 
relevant pubW sch^bols were spotted and the school authorities were 
contacted with regard to sampling children in the first grade. Once 
classes were identified, our examiners sampled as randomly as pos- 
sible the children' at these various locations. Children with physical 
handicaps or emotional disorders, or who became ill for more than a 
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week were discarded/ ^so, the results of children whose testing was 
"requested" were onAfted. Appf^imately 500 children were either 
completely or partially tested ia brdcT to try to maintain a fairly ran= 
dpin selection within ouf sHratifi^'samplp. ^* 

^ ' * ^^-''^ ^ ' 

Thv SraiM ^ ^' ' 

The scales used in this study are 'rather extensive, modifications of 
scales previously developed under a U.S. Office of Education grant , 
conducted at Hunter College froLm 1957 Xo 1960/T^ four scales ar&i 
^^erbai Reasoning, Numerical and Space, The Verbal scale consists 
^f a 304tem picture vocabulary^ tek and two 1 S-item .word definition 
tests yielding two-part scores; tha^s, a Picture Vocabulary score and 
a Word Vocabulary score' plus' a total score. Both of these su^4ests 
are scored on the basis of two points, for each correct or completely 
acceptable response and one point for partially correct responses. The 
scoring keys were careful^ developed by thfc group ^f psychologists 
participating in the study, ^ ^\ , ) ^ 

The Reasoning scale consists of three tests; Picture Analogies,, Pic- 
ture Arrangements apd Jump Peg, The first two tests are ralati^^eiy 
^ell-known types and no- major deviation^ ffom, the mm\ fcirifial is 
employed. The Jump Peg test, however, is^ quite novel in that^the task 
required of the subject is unlike that of any of the children's james 
available on the market, although somewhat ynlilar. The administra- 
tion of this subtlest is planned to enable each thlld to start from th^ 
same point of knowledge about the task. It is feU that the test is a 
good measure of reasoning whenever perceptual variance qan be, 
neutralized, ' ' , , 

The Numerical scafe consists of five subtests: Etiumerittion — ^5 
items; Addition — 10 items; Subtraction — ^0 items; MuUiplication 

^10 items; Division — 10 items. Knowledge of neither numbers nor 

operational symbols is required on any of these te^ts. All material is 
presented pictorially and with the gxception of the Enumeration test, 
in which t^e subject i? permitted tovcount with his fingeraouching the 
picture, all of the numeJfical operations have to be done in the child's 
head/ It is recognized that this scale is quite unlike the usual number 
factor tests. ; K ' _ 

The Space scale consists of 4 sub-tests: Object Cornpletion; Jig- 
Saws; Estimating Path; and Perspective. The Object Completion sub- 
testis more a perceptual-visualization measure than a pure space 
measure, but this^ype of test has shown to correlate fairly well with 
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^ threi^dimensi6nal/ineasur&, TM Jig Sa^ consist of puzzles^ quite ^ 
/Ainlike those availkble in chil^fc|amei^he requi^p the tubj^t^ 
;^rceivi spayal rela^onshi^^Bpi^ to corftplete Hhem spccessffflly , ^ 

but require Ifttfa motor proffllilWyand cohtain none of the usual rpa- ' 
soning c^i fdund in/popular j^-sa^ p^l^s> The EsUmating Path ^ 
tfe^ was adapted from spatial tests y^etfin the Air Force and prob^^ 
. aWy is the weakest test in -our Jbattery owing to> its susce^Hbility Jo 
^chance^he task of the test is to estimate visually projecftd^gWis 
.airplanes, thp^ers^ctive sub^testpbn^isfs of identifying Ul^tfld oT^^ 
;vjsiqp ofi ^veral persons at various ppints in a scene. ^Aljtmygh it is% 
fdt tto the hypothetical construcuunderlying tfiis test is priimr^y^ 
spati^t it is^admitt^nKat the meth9^ used to evaluate ihe cops^^ 
possibly uitroduces^yefbal and realoning jariance at^the &p/>yel 

studbd. ^ ^ , L' _ ' ^ . * ■ ■ \] ' - 

aSn^e general consmrations -about the construction of the tests 

shd^^enoted| '^l^^ ^ 

■ 1. No i^'^ding was required of tjie subject on any test, 
sj,- All pictorial materials ^ere clearly pfesented on large cards for 
easy viewihg, * - \ 

3. All persons' were* drawn as neutral as possible.^ith a careful 
..avoidance of the- precious, pretty pictures of (he usual chiU 
dren's^cture^bpoks, c . 
. 4. ' No itlkf required the nar^ffg ^f an object by the subject except 
on ^he Verbal scale. The addltionarexception to this was the 
Object Completion tfsl; hov^vc^; the subject could obtain full 
credit without namit^an object if he could indicate his recog= 
V nit ion in some other way. 

L 'An attempt was jnade to includ^ only elements which appear 
commonly in all cultural and class groups in New Work City. 
. Table 1 ^ntains the Characteristics of the Tests and the Statistics 
Obtained by the Total Group. It can be noted that the rcliabiliUys of 
the four total scalesW Verba} — .93, Reasoning — .92, NutnencaT 
— .96, and Space — Rcasi)nably high reliability coefficients were 
obfained for all of the fourteen sub-tests with the exception of Esti- 
mating Path/ Perspective, and thd 6-itcm Enumeration sUb^test. The 
i intercorrelatibns of the four pain scales are as follows: Verbal = 
Reasoning, .58; Verbal — NumelcaU J4; Verbal — Space, .44; Rea- 
soning—Numerical, .73; itcasdhing — Space, .62; and NumeHcal 
. =i.Spw, ^54. As you can see, these are higher than one would like 
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31,0 
74.4 

7J 
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them t^.B.e, Fortunately, the high rejiabilitics enable us to ass^ 
sufficient. Unique variance to justify utilizing these scales to differen- 
tiate thesfe four ateas'of meTital aM 
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Testing Fraosdures 

Four psyphologists helped with the reviMon of the scales and admin- 
istered the tests. Om was Jewish, one Negro, one Puerto Rican^ and 
one Chinese, Each had been trained beyond the master's degree level 
and each had ample experience ^itimijiistering psychological tests, in- 
cluding the standard tests^of intelligtnce. ,Each of the 320 children In 
our study 'was tested by a psycfioni^trician who shared his cultural 
identity. Whilfc the length of the testing period varied and was deter- 
mined primarily^by the examiner's judgmerrt regard^g degree of rap- 
port and fatigue, niost commdnly a child was seen for one session on 
each of three separate days for 30 to 45 minutes. 

The test required neither reading nor writing ability and directions 
were kept extremely simple. The test could be administered in Eng- 
lish or in the ghlld's primaiy language, or in a combination of the 
two; children were permitted to respond in any combination of Ian- 
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guages, ^Ample practice material ^was provided prior to each S^fl^iJ 
and examners were Lnstructed to proceed with scored Items only when 
•/»/ eferti^ that the child was familiar with the tnaterial and understood. 
>-^.tB^k. ^ ^ .. - . ■ ■ 

£:^*^^^tp''^ttQ fto formal steps of test administration left to the dis- 
^eti& prttlie examiner; each step was specifled in detail to assure 
^^^"II ft^Hlfd jr^cedure. Each examiner saw every other examiner and 
Rlroserirafenister the test to a fchild (during the gretest phase) on 
- vidiQ tape: Each examiner also tested' 1 child before a one-way visions 
' sereen while the other three took careful notes of the most minute 
^ - ^Meviation from standard procedure. ' ^ * * 

With four exceptions, ea<^ fchild%as tested in his own school, in 
a room alone widi die examiner. TExtolners pfcsented the tasks with 
.a **game" orientation, but were iristTAicte4#to change the orientation 
to a serious one if the child seemed more Qemfortable and praductive 
when dealing with ''work,"- ' 

Class^and Bthnle Oroup &#slgnat|an 

The designation of socio-economic class provided one of the major 
^ problems in'the study. To make a^very long story short, the ultimalp 
technique employed was an adaptation of the Hollingshead and 
Redlich scale. The three factors of father's (that is, head of the house- 
hold's) occupation, father's education, and dwelling unit were rated, 
A composite was then computed resulting in ti 5-ciass scale. Classes 
I, II and iH were designated middle class; .Classes IV ahd V lower 
class* 

A major limitation of our study is the differential level of the class 
designations of the subjects in the four ethnic groups and the differen- 
tial separation of the class designations within the various groups. For 
example, approximately two-thirds of the Jewish middle-class sub- 
jects were in Class I on our scale. The Chinese and ^Icgrp middle-class 
subjects were predominantly Cifiss IL The Puerto Rican middle-class 
subjects were in Classes II and III. The Jewish lower-class subjects 
were equally divided between Classes IV and V, whereas the other 
= three lower-class groups were almost cntir^ in Class V. Hence, aU 
though we acliieved good class scparaliun -^hat is, at least one class 
positibn in our scale was skipped by each ethnic group — the sepa= 
rations did not occur at equivalent points on the scale. However, this , 
situation is the reality in New York City, and we would have distorted 
our population by sampling to obtain equated groups. ^ 
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The ethnic designation was somewhat simpler on paper. The par- 
ents of each subject had to be known to be of the particular ethnic 
group, The only problem w^s the actual determination of the facts 
for some of the subjects; Cases were discarded, however, if verification 
of the parentage could not be obtained. 



Results 



The analyses of the data are too extensive to be presented here in 
any detail, but certain highlights^ should prove of Interest, The sup- 
port or lack^of support for Our three hypotheses can be seen in Table 2 
■and the grapWc representation of them in Figure 1 , 



Means @f Normalised Standard Scares on the Four Ss&les 
Lleled by Ethnjc and Soolal Clatts tBrsup ^ 



Group 

Jewish Middle Cld$s 
, Lowef Ciss! 



Total 



Chinese Middle Cldsi 
Lower C\Qi% 



Totdl 



Negro Middle Closi 

Lower CIdss ^ 
Tofol 

Puerto Rksn Middle Closi 
Lower Ciesf 
Total 

Totqi Middle Closi 
Lower Clasi 



Verbs 1 


Resionlng 


NumejIcQl 


Space 


62.6 




59,2 . 


56.5 


547 


4B.5 


50.2 


47,0 


SB J 


52.6 


54.7 


51,8 


51.2 


56.0 -y 


56.0 


56.S 


45.2 


51,8 


51.8 


52,0 


48,2 


53.9 


53,9 


54.4 


55J 


53,9 


51.2 


53.0 


44,0 


4h5 


39.6 


40.1 


49.9 


47.7 


45.4' 


46iS 


47.4 


48.7 


49.2 


49.4 


39,3 


42.5 


4310 


45.3 


43.3 


45.6 


46,1 


47,4 


54,2 


.53.8 


53.9 


53.9 


45,8 


46.1 


46.1 


46 J 



Our first hypothesis, that significant^ differences exist among groups 
of children from different spcial class and cultural backgrounds' in 
each of the four mental ability^ areaSj was supported. § 

Our second hypothesis, that significant differences exist among 
groups of children from different social class and cultural backgrounds 
in pattern or configuration of scores from these diverse areas of mental 
ability, was partially supported. ( 

Out third hypothresis^ that significant interactions exist between the 
variables of social class and cultural background In determining the 



level of each meqjal ability and the riatUre of the patterns among 
them, was partially supported. - ^ ^ - 

. One can see from the graphic representation of the means that 
sociai class seems to influence level of functioning, and culturkriden- 
tity seems to^'influence the' parrel, or relative highland low ability 
areas. We have no evidente that these two factors interact in any way ^ 
that creates significaiit differflices in functioning, , - ^ 

One Of two points should be emphasized; Note that the lowest ^mean 
group score is earned by the lowen-class Puerto Rrcgns on Jhe verbal 
tasks. Yet this same group does quit^ well, compared with fhe lower- * 
class Negroes, on spafial tasks. / 

There is a clear and almost uniform^ diTlTei^ence within each= cul- 
tural group that is imposed by social class and runs Vi^rass tjie tour 
mental ability areas. Yet, notice the difference in degree of separa- 
tiort between the two social classes of Chinese and jhe two social 
classes of Negroes. To.be su^e^ this may be partly due fo factor^ built 
into our ^udy but it still suggests the strong possibility that social-class 
difference will more strongly aflect one's identity and intellectual per- 
formance if he is. a Negro than if he Is Chinese. ^ * 

It hiay be=wise to point out that these data refer to differences in 
the performance of groups and not the performanceif of imUvkhiah. 
They do not indicate how an individual will perform, but if he belongs 
to one of these eight groups, tl^cy suggest how he is likely^ to perform. 
His individual deviation, be it high or low, on one score or another* 
thereby becomes more useful information in understanding his indi- 
vidualabilities. 

One approach to utilizing te^t data such as these for identifying 
partiduiar patterns Jor individuals is a classification technique sug- 
gested by Tatsuoka, Table 3 illustrates the application of the method 
to our data. Note that the middle-class Jewish children and the lower- 
class Negro children were most accurately classified and the lower 
class Puerto Rican^ children lightly less so. Note, hovv^ever, that in 
addition to the 20 Ptierto Rican lower-class children typical of their 
class and ethnic group pattern, an additional 15 had patterns like 
other lower-class patterns (that is, Chinese and Negro ^ It is apparent 
that the middle-class Puerta Rican children were the most hetero- 
geneous of any of the eight groups. ' ' , 
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Oraup 

MIddl* Oilfisss 
' Lewer Ctrintie 
MM^ Jtwfih 

Middis Htfirp 
Uwer Nvgre 

Liswtr fQ«rtd lOcsn 

•RoiiF*! t@ b« fend dEr©» a* fell^wu The iqspei ef 1 3 mlddle^daii Chif^je iub^«i fit 
^.f*«i»l*U«.dTO-ChfaiMapdtt»m Ityel on the fQur_mtntoI ability s|plei| 10 mlddle^G aii 
Chiiiai» feok iPpf « HkeJowap-clau Chinese; 6 Iqsk mspe like middle-crass Jews, 1 mare like , 
Q 1ew9r*clsi« Mt, et€. 



It jiapparent that our sfudy has raised mor^ questions than it has 
proc^i^ neat answers; Our tests need revision and refinement, Addi- 
tldnia analyses currently being completed of the factorial composition 
^.of th©t?attary and of the unique non-chance variance of each subtest 
will ^ useful in further interpretation of our data. It is believed that 
we have strong evidence of differential patterns of mental abiUties 
amortg four New York City ethnic groups. The data reveal sharp test- 
perfcTOiance differences between middle^ and lo\^Mlass groups re-^ 
garaiess of ethnic group and that the differences differ among the 
.ethnic groups. Replications of the study on thc^ same and additional 
ethnic groups using psychomeiricians of like and unlike background 
would be valuable, There is some evidence of differences in patterns 
.among..thi grQUps that j^^ cqnsidurabte value in educational 

planning. The value of the study will be finally realized in the follow-up 
studies we hope it will stimulate, - 
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siriGEL, ARTHUR, Applied Psychological Services 
siLVA, MILTON N., CoHege Entrance.Examination Board, Puerto Rico 
siLVERBERG, EDWARD B., New York City Department of Personnel 
siNOLETON, CARLTON M., Wesleyan University 
SKAOER, RODNEY w.. Educational Testing Service 
SMITH, ALEXANDER F., Southem Connecticut State College 
SMITH, ALLAN B., Rhode Island College 
SMITH, ANN z,. Educational Testing Service 
SMITH, MARSHALL p„ Treuton State College 

SMITH, MARION s,, South T^nth Street School, Newark, New Jersey 

SOAR, ROBERT s.. Temple University 

SOLOMON, HERBERT, Stanford University 

SOLOMON, ROBERT, Educational Testing Service 

soMMER, JOHN, Houghton Mifflin Company 

SONTAG, MARVIN, Ncw York City Board of Education 

sosuLsKi, M. SL John Fisher College 

SOTHERLAND, DONALD w., Ncw York Department of Civil Service 
SOUTHER, MARY T., Towcr Hill School, Wilmington, Delaware 
SPANEY, EMMA, Queens College of the City University of New York 
sPAULDiNG, GERALDiNE, Educational Records Bureau, Winston-Salem, 

North Carolina ; ? 

SPEER, GEORGE s., Illinois Institute of Technology 
SPENCER, RICHARD E., University of Illinois 
SPREMULLi, ESTELLE, EducationyJ Testing Service 

SQUIRES, JO ANNE s., Hamplou Roads Speech and Hearing Center, New- 
port News, Virginia ^ 
^ STAHL, SUZANNE, Educational Testing Service ^' 
STAKE, ROBERT E., University of Illinois 
STAGEY, JULIAN c, University of Wisconsin 
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STATLER, CHAitEi R., UDiversity of Jowa 

STEiNMAN, ARTHUR M., Tranton Stale College -^r 

STELLER, DANIEL J., PaCa Cotltge 

STEPHEN, SISTER MARY, Villa JuHe College 
STEPHENS, ROBERT w., Elma, Washington Schools 
STEV/ART, CLIFFORD, University of South Florida 
STEWART, E. ELIZABETH, Educational Testing Service 
STEWART, NAOMI s., Cranford, New Jersey 
STiCKELL, DAVID w„ Bucknell University . = 
STILES, GRACE ELLEN, University of Rhode^sland / 
STIMPEL, SIDNEY, Martin G. Johnson Reporting Company 
STOKER, HOWARD w>, Florida State University / 
STOLTZ, ROBERT l., ScieHce Research Associates 
STONE, PAUL T., Huntingtfon College 
STONE, SELMA, The City College of New York 
STOUGHTON, ROBERT w., Connecticut State bepartment of Education 
STRiCKER, LAWRENCE j.j Educational Testing Service 
STUDDiFORD, WALTER B,, Prlhceton Uniyersity 
STUNKARD, claVton L., University of Maryland 
SUPER, DONALD E,, Columbia Uniyersity 
sussMAN, LEAH, Newark (New Jersey) School System 
iSUTTON, JOSEPH T., Stelson University 
SWAN, BEVERLEY bT, Florida State Department of Education 
SWANSON, EDWARD b>, University of Minnesota 
swiNEFORD, FRANCES,' Educational Testing Service 
TABER, VICTOR A., T^ew Yofik>State Department of Education 
TANNER, DANIEL, The City .Uinivefsity of New York 
TATSUOKA, MAURICE, University, of Illinois 
TAYLOR, BRUC|, Edu||Uonal Testing Service 
TAYLOR, JUSTiNE,,^fflpUonal Testing Service 
TAYLOR, SAMUEL ^^^ew Vork State Civil Service Departmqnl 
TCHORNi, BERNAR^ feAifiaUonal Testing Service 
TERRAL, JOSEPH ^,^|Li^fional Testing Service 
terWilliger, JAMES S t'^Peubody College 
THANOS, cvNTHiAt>The Qty .College of New York 
THEVAps,DEi^o^ The P|||S^rvania State - 
THOMAS, wiujAM "f ., l^versity of Wisconsin 
THOMPf^, R|YMd^p E.f Educational Testing Service 
TH0K^6Ni!MURiEL, JLfca Abgel^s, California 
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TRiooSi FRANCES, Committeepn Diagnostic Reading Tests, Inc. 
TRiSMEN, boNALD, Educatioily Testing Service 




TUCKER, DONALD K,, Northf aslb'n Univtiiity ^ 
TUCj^ER, LEDYARb R, Univeriity of Illinoii ' 
TUPTEi CORINNE Q., Teacher College Gomputer Center, New York City 
TULLY, EMERSON^ Florida Stat© University 
TURHBULL* WILLIAM w., Educationa! Testing Service 
TYLER, GLADYS K.^ Psychological Services, Hamden, Connecticut, 
UMANS, SHELLEY, New York City Board of Education 
UNOIR, R.j New York, New York . 
URQUHARt, HELEN L., Brown Uniyersity J 
UKRY, VERN w., U.'S. Army Enlisted Evaluatidii Center / 
VALLEY, JOHN R,, Eopcational Testing Service ^ ? 

VAN aUsdall, LEIGH, ^cience Research Associates j 
VANiCA, SUMNER, Akrqp, Ohio / 
VECCHIONE, NICHOLAS, Educational Records Bureau / 
viscEGLiA, JOHN A., Camden (New Jersey) Board of Educatiph 
vmLis, MORRIS s,, University of Pennsyivania ^ / 
VOLANTE, WILLIAM, The Plngry School, Elizabeth, New Jersey 
VON FANQE, ERICH A., Concprdia Lutheran Junior College / 
VRiEND, JOHN, New York University , ? 
WAGNER, EDWARD P., Bloomesburg State College, Pennsylvani^i 
WAdNER, MAZiE EARLE, State University of New York at Buflfalo 
WAHLOREN, HARDY L., State Univenllj* of New York at Geneseo 
WAiDLEY, JOHN w., New York University ■ 
WALKER, HELEN M., Teachcrs dollege, Columbia University 
WALKER, ROBERT N., Personnel Press, Inc., Princeton, New Jersey 
WALL, VICTOR D., Lajunta (Colorado) High School 
WALLACEi wiMBURN L., The Psychological Corporation 
WALSH, JEANNE, Washington Square College 
WALTHEW, JOHN K., Educational Testing Service 
WALTON, WESLEY W., Educational Testing Service 
WALTZMAN, HAi., New York State. Department^^feivil Service 
WALWER, FRANK K,, Columbia University 
WANTMAN, MOREY, Educational Testing Service 
WARD, ANNIE w,, VolusiaCounty«( Florida) Public Schools 
WARE, DOROTHY A,, Detroit Psychological Clinic 
WARRINGTON, wiLLARD G,, Michigan State University 
WATERSi^MAX, The Pennsylvania State University; 
WATKiNS, RICHARD w.. Educational Testing Servide 
WATTj ROBERT J., Glenbrook NortlrHigh Schoo!, Northbrook, Illinois 
WEBER, 'BURTON J,, Ridgeficld tNew Jersey) Memorial High School 
WEBER, EWALD, New York City Department of Personnel 
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WECHSLERi DAVID, Ballevue Psychiatric Hospital, New York 

WEINBERG, CARL, University of Delaware 

WEiNEks MAX, Broeklyn College 

WEiiBRDDT, JEROME, Purdue University 

WE ITZ, HENRY, Duke University - ^ 

WELNA, CECILIA, University of Hartford ^ 

wESMAN, ALEXANDER 0., The Psychological Corporation 

WEST» LEONARD J., The City University of New York 

WHALEY, FRANCIS L,, The Pennsylvania State University 

WHETSTONE, ROBERT D./Univcrsity of Colorado 

WHITE, HERBERT L., New York City Department of Personnel 

WHiTLAj DEAN K., Harvard University 

WIENER, ROSA D., Learning Institute of North Carolina 

WILDE, GERRIT J., Institute of Clinical and Industrial Psychology, Holland 

WILDER, DAVID Hv, Bucknell University^ 
* WILDER, OITA, Edueational Testing Service 

WILDS, PRESTON LEA, Medical College of Georgia 

WILKE, MARGUERITE MrrHuntef College 

wiLKE, WALTER H., New Yofk University 

wiLLARD, RICHARD w,, Massachusetts Institute of Technology 

WILLEY, CLARENCE F., Norwich University * 

WILLIAMS, CLARENCE M., University of Rochester 
^WILLIAMS, E, BELViN, Teachefs tollege, Columbia University 

WILLIAMS, GERALD, The Pennsylvania State University 

WILLIAMS, IRENE, Educational Testing Service 

WILLIAMS, JOHN M., Educational Testing Service 

WILLIAMS, MALCOLM J,, United States Coast Guard Academy 

WILLIAMS, ROGER K,, Morgan Stale College 

WILLIS, WARREN K.^ University of Illinois 

WILSON, MARGARET, Educational Testing Service 

wiNGO, ALFRED L,, Virginia State Department of Education 

WiNiEWicz, CASIMERE s,, United States Naval Examining Center 

wiNTERBOTTOM, JOHN A,, Educational Testing Service 

wiRSio^ JANE, Educational Testing Service 

WODTKE, KENNETH H,, The Pennsylvania State University ^ 

woHLHUETER, JAMES F., Educational Testing Service 

WOLF, RICHARD, University of Chicago 

WOLINS, LEROY, lowa State University 

WOLMANi BENJAMIN B,, Adelphl University 

WOMER, FRANK B., University of Michigan 

WTOO, BEN D., Columbia University 

WRIGHT, DANIEL w., New Vork University 

BRIGHT* WILBUR H., State University of New Vork at Genesco ^ 
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zoOK, DONOVAN Q., Board of Examiners for the Foreign Service 
zUBiN, JOSEPH, Department of Mental Hygiene, New York 
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